-- Logs begin at Wed 2022-09-14 11:10:52 UTC, end at Wed 2022-09-14 12:34:50 UTC. -- Sep 14 11:44:28.834585 dv-cinder-844962-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 14 11:44:28.844730 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 14 11:44:28.846765 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 14 11:44:28 2022] *** Sep 14 11:44:28.846765 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 14 11:44:28.846765 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 14 11:44:28.846765 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: nodename: dv-cinder-844962-2-iscsi Sep 14 11:44:28.846765 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: machine: x86_64 Sep 14 11:44:28.846765 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: clock source: unix Sep 14 11:44:28.846765 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: pcre jit disabled Sep 14 11:44:28.846765 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: detected number of CPU cores: 4 Sep 14 11:44:28.846765 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: current working directory: / Sep 14 11:44:28.846765 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: detected binary path: /usr/bin/uwsgi-core Sep 14 11:44:28.847340 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: your processes number limit is 47680 Sep 14 11:44:28.847340 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: your memory page size is 4096 bytes Sep 14 11:44:28.847340 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: detected max file descriptor number: 2048 Sep 14 11:44:28.847340 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: lock engine: pthread robust mutexes Sep 14 11:44:28.847340 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: thunder lock: enabled Sep 14 11:44:28.847340 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 14 11:44:28.847340 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 14 11:44:28.886571 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: Python main interpreter initialized at 0x556f412cd9c0 Sep 14 11:44:28.886571 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: python threads support enabled Sep 14 11:44:28.886571 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: your server socket listen backlog is limited to 100 connections Sep 14 11:44:28.886571 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: your mercy for graceful operations on workers is 90 seconds Sep 14 11:44:28.886886 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: mapped 403077 bytes (393 KB) for 2 cores Sep 14 11:44:28.886988 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: *** Operational MODE: preforking *** Sep 14 11:44:28.887264 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: *** uWSGI is running in multiple interpreter mode *** Sep 14 11:44:28.887264 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: spawned uWSGI master process (pid: 81569) Sep 14 11:44:28.887449 dv-cinder-844962-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 14 11:44:28.888266 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: spawned uWSGI worker 1 (pid: 81581, cores: 1) Sep 14 11:44:28.888354 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: spawned uWSGI worker 2 (pid: 81582, cores: 1) Sep 14 11:44:28.889914 dv-cinder-844962-2-iscsi devstack@keystone.service[81569]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 14 11:44:33.209492 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 11:44:33.209492 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: from cryptography.utils import int_from_bytes Sep 14 11:44:33.210335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 11:44:33.210335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: from cryptography.utils import int_from_bytes Sep 14 11:44:33.211338 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 11:44:33.211338 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: from cryptography.utils import int_from_bytes Sep 14 11:44:33.211792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 11:44:33.211792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: from cryptography.utils import int_from_bytes Sep 14 11:44:33.264241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 14 11:44:33.264419 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 14 11:44:33.264503 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 14 11:44:33.264620 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] command line args: [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 14 11:44:33.264707 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 14 11:44:33.264822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] command line args: [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 14 11:44:33.264900 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 14 11:44:33.265063 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ================================================================================ {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 14 11:44:33.265141 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 14 11:44:33.265256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ================================================================================ {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 14 11:44:33.265346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] admin_token = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.265510 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] admin_token = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.265602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] config_dir = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.265736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.265821 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] config_dir = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.265973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] config_source = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.266061 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.266138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] config_source = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.266214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] debug = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.266321 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] debug = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.266401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.266672 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.266796 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.266899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.266899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.267078 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.267183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.267288 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.267389 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.267489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.267592 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.267692 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.267803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_config_append = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.267898 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_config_append = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.267996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.268091 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.268190 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_dir = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.268281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_dir = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.268386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_file = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.268487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_file = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.268593 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.268698 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.268819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.268895 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.269018 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.269099 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.269170 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.269244 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.269347 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.269476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.269563 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.269641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.269755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.269836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.270041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.270149 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.270283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.270367 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.270529 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.270617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.270753 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.270888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.270971 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.271053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.271216 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.271334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.271465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.271564 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.271738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.271822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.272226 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.272322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.272403 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] publish_errors = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.272480 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] publish_errors = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.272628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.272722 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.272856 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.272942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.273034 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.273118 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.273278 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.273431 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.273533 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.273609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.273735 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] standard_threads = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.273957 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] standard_threads = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.274073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.274152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.274229 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.274312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.274390 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.274469 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.274546 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] use_journal = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.274621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] use_journal = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.274715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] use_json = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.274824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] use_json = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.274900 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] use_stderr = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.274982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] use_stderr = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.275058 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] use_syslog = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.275137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] use_syslog = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.275336 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.275420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 11:44:33.275498 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.275601 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.275737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.275815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.275996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.276079 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.276236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.276390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.276487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.276569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.276721 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.276798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.276970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.277059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.277207 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.277285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.277451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.277610 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.277700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.277787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.277982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.278074 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.278220 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.278297 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.278464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.278550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.278700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.278777 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.278948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.279031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.279197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.279281 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.279431 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.279507 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.279684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.279761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.279941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.280018 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.280184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.280259 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.280422 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.280504 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.280674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.280750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.280937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.281014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.281188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.281269 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.281963 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.282060 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.282206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.282293 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.282441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.282523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.282664 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.282736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.282896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.282974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.283146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.283226 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.283392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.283469 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.283624 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.283703 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.283856 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.283939 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.284104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.284182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.284353 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.284433 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.284592 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.284669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.284839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.284916 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.285070 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.285152 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.285305 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.285380 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.285541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.285623 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.285771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.285846 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.289347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.289457 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.289457 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.289583 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.289642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.289725 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.289802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.289894 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.289974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.290036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.290094 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.290151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.290208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.290265 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.290332 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.290392 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.290455 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.290513 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.290570 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.290634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.290693 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.290760 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.290940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.291011 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.291076 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.291137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.291195 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.291263 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.291329 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.291452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.291529 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.291643 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.291718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.291846 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.291920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.291990 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.292112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.292188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.292259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.292385 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.292461 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.292527 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.292586 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.292726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.292815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.292897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.292970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.293090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.293165 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.293236 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.293356 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.293439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.293513 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.293627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.293701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.293814 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.293905 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.293991 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.294065 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.294185 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.294260 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.294328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.294441 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.294562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.294645 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.294767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.294848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.294921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.295034 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.295181 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.295256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.295321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.295397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.295469 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.295582 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.295671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.295750 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.295822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.295892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.296007 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.296091 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.296171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.296251 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.296324 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.296394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.296464 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.296534 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.296634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.296720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.296794 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.296865 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.296935 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.297006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.297006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.297869 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.297956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.298086 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.298171 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.298244 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.298316 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.298394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.298472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.298544 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.298614 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.298685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.298755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.298834 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.298906 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.298977 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.299138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.299219 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.299370 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.299456 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.299606 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.299693 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.299840 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.299947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.300083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.300218 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.300320 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.300393 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.300570 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.300928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.301062 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.301152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.301320 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.301408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.301576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.301668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.301816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.301924 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.302093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.external = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.302247 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.302393 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.302520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.external = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.302520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.302760 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.302887 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.303019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.303151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.password = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.303274 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.303399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.token = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.303522 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.password = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.303645 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.303776 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.token = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.303901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.304035 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.304158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.304282 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.304406 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.304538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.304666 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.304790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.304913 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.305036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.305161 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.305292 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.305417 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.305540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.305664 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.305801 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.305949 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.306086 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.306212 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.306335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.306459 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.306582 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.306704 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.306835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.306958 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.307079 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.307203 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.307326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.307458 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.307588 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.307717 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.307929 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.308051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.308132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.308205 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.308286 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.308359 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.308430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.308501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.308572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.308718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.308799 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.308872 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.308944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.309021 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.309094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.309177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.309251 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.309322 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.309421 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.309528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.309669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.309756 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.309949 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.310032 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.310187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.310266 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.310437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.322478 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.322571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.322636 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.322704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.322779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.322852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.322938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.323015 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.323088 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.323167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.323242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.323314 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.323394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.323468 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.323541 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.323541 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.323671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.323743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.323840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.323916 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.323994 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.324068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.324140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.324212 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.324291 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.324365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.324437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.324508 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.324580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.324651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.324730 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.324804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.324876 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.324948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.325020 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.325092 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.325172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.325251 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.325325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.325397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.325469 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.325546 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.325628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.325703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.325776 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.325848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.326007 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.326083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.326168 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.326243 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.326316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.326388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.326460 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.326532 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.326616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.326690 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.326762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.326834 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.326906 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.326990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.327066 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.327138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.327210 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.327282 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.327354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.327445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.327521 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.327593 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.327665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.327736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.327808 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.327887 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.327967 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.328040 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.328112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.328183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.328255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.328916 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.328997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.329070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.329142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.329215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.329286 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.329366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.329440 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.329513 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.329584 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.329656 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.329727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.329808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.329909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.329986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.330059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.330130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.330202 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.330286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.330362 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.330434 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.330505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.330576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.330648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.330727 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.330801 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.330873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.330945 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.331016 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.331087 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.331167 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.331241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.331313 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.331385 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.331456 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.331528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.331608 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.331682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.331755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.331831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.331903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.331975 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.332060 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.332135 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.332208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.332279 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.332352 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.332423 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.332502 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.332576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.332648 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.332720 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.332800 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.332874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.332954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.333028 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.333100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.333171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.333242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.333313 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.333394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.333768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.333902 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.333982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.334052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.334113 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.334187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.334250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.334309 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.334368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.334433 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.334433 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.334548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.334610 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.334669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.334728 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.334787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.334845 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.334911 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.334973 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.335032 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.335090 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.335150 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.335150 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.335268 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.335268 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.335375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.335435 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.335493 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.335552 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.335617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.335678 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.335737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.335795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.335853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.335911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.336062 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.336127 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.336186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.336244 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.336302 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.336360 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.336427 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.336488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.336546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.336604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.336663 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.336721 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.336787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.336848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.336907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.336965 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.337023 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.337089 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.337150 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.337208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.337277 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.337337 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.337395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.337461 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.337521 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.337579 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.337638 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.337696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.337754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.337819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.337907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.337982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] role.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.338062 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.338122 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] role.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.338180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.338246 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.338307 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.338365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.338424 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] role.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.338482 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.338540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] role.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.338605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.338666 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.338724 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.338783 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.338841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.338900 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.338966 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.339027 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.339085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.339144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.339203 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.339261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.339326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.339387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.339445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.339504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.339562 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.339620 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.339686 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.339746 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.339804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.339863 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.339863 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.339985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.340053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.340114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.340200 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.340264 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.340264 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.340368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.340435 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.340502 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.340562 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.340621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.340679 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.340745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.340806 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.340864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.340922 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.340979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.341038 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.341038 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.341246 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.341368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.341482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.341609 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.341729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.341817 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.341918 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.341994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.342066 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.342205 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.342288 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.342444 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.342528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.342682 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.342764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.342914 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.343002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.343152 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.343235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.343388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.343469 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.343630 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.343709 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.343865 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.343972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.344645 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.344749 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.344889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.344999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.345142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.345243 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.345383 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.345486 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.345643 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.345722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.345922 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.346008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.346168 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.346250 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.346415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.346495 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.346658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.346738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.346905 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.346985 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.347143 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.347222 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.347385 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.347466 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.347655 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.347747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.347885 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.347964 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.348140 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.connection = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.348227 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.348382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.348462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.348627 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.348720 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.348865 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.348945 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.349105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.349193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.connection = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.349339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.349420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.349575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.349655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.349809 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.349949 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.350068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.350149 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.350312 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.350392 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.350547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.350698 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.350775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.350848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.351014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.351095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.351260 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.351413 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.351515 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.351595 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.351725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.351805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.351954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.352043 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.352194 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.352275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.352428 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.352508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.352601 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 14 11:44:33.352734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.352972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.353217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.353456 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 11:44:33.353629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 14 11:44:33.381285 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.381513 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.381644 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.381730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.381908 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.382132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.390015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.390466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.418624 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.418856 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.419619 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.419855 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.478662 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.478843 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.527944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.528172 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.528993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.529234 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.699646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.699882 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.700099 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.700304 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.700397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.700528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.700611 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.700745 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.700838 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.701003 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.701206 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.701428 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.731121 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.731335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.731475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.731566 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.731715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.731800 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.731948 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.732041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.732162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.732244 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.732379 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.732461 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.732600 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.732821 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.743442 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.743909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.758721 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.758838 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.758934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.759038 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.759166 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.759257 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.759393 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.759484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.759615 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.759697 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.759832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.759913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.760047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.760130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 11:44:33.777172 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.777381 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 11:44:33.777472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.777660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 11:44:33.777761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.777937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 11:44:33.778025 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.778213 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.778296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 11:44:33.778475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.778560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.778734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.778820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.778994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.779076 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.779260 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.788513 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.788789 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.788976 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.789059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.789257 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.789341 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.789534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.789822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.796238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.797115 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.799163 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.799449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.799733 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.800017 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.800106 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.800311 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.800395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.800574 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.800922 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.801110 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.801201 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.801385 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.801466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.801651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.801731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.801954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.802083 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.802273 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.802369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.802529 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.802809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.803081 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.825231 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.826512 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.826721 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.827011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.827999 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.828334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.832222 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.832504 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.832786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.833563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.833876 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.834158 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.839094 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.839384 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.840520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.840811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.845091 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.845379 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.845664 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.845976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.846255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.846530 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.846688 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.846800 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.846998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.847094 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.847290 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.847574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.847874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.848150 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.848427 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.848722 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.862487 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.862765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.864343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.864624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.866610 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.868453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.869808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.870135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.871820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.872120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.874323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.876804 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.878038 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.879976 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.881437 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.881514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.881782 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.882090 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.882372 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.882659 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.883295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.883600 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.883891 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.884177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.884458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.884820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.898887 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.899173 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.899457 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.899703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.901360 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.901650 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.901946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.902216 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.906535 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.908095 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.909143 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.909605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.909901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.910733 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.912318 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.912514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.912602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.912808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.913091 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.913371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.913751 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.915289 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.915587 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.915923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.916228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.916627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.924887 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.925178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.925463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.926041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.926425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.926718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.927006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.928157 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.928445 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.928730 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.929016 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.929408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.929693 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.930024 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.942731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.945629 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.946078 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.948510 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.949112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.950172 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.951396 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.951680 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.952089 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.952171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.954297 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.955777 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.956065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.956522 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.958933 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.960365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.963030 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.963263 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.963659 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.963946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.964226 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.964497 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.967400 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.967805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.968174 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.968433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.972051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.972341 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.972620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.976391 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.976687 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.976972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.979461 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.979749 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.980029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.980310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.984595 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.984595 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.984595 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.984595 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.984595 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.984595 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.984960 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.984960 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.984960 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.984960 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.985207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 11:44:33.985207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.985207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.985207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.985207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.985492 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.985657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.985962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.986238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.986527 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.986811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.987086 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.987352 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:33.987672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 11:44:34.017661 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556f412cd9c0 pid: 81581 (default app) Sep 14 11:44:34.020979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9d771-3acf-4588-9d12-8acbe5fa21f9 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:34.021265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9d771-3acf-4588-9d12-8acbe5fa21f9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:34.021543 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9d771-3acf-4588-9d12-8acbe5fa21f9 None None] PATH_INFO: `/v3` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:34.023053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556f412cd9c0 pid: 81582 (default app) Sep 14 11:44:34.025658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1/1] 10.222.0.62 () {50 vars in 791 bytes} [Wed Sep 14 11:44:34 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:44:41.636340 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869babe-6a3b-4dd9-b843-e917efab6c8d None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:41.636884 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869babe-6a3b-4dd9-b843-e917efab6c8d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:41.636884 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869babe-6a3b-4dd9-b843-e917efab6c8d None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:41.641673 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1/2] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:44:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:44:41.647374 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743456d2-70e3-4b51-a4ad-62630be33b67 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:41.647787 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743456d2-70e3-4b51-a4ad-62630be33b67 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:41.647931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743456d2-70e3-4b51-a4ad-62630be33b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:41.717755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: INFO dbcounter [None req-743456d2-70e3-4b51-a4ad-62630be33b67 None None] Registered counter for database keystone Sep 14 11:44:41.736871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG oslo_db.sqlalchemy.engines [None req-743456d2-70e3-4b51-a4ad-62630be33b67 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81581) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 14 11:44:41.743033 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writer thread running {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 14 11:44:41.920717 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG passlib.handlers.bcrypt [None req-743456d2-70e3-4b51-a4ad-62630be33b67 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81581) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 14 11:44:41.921039 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG passlib.handlers.bcrypt [None req-743456d2-70e3-4b51-a4ad-62630be33b67 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81581) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 14 11:44:41.978966 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-743456d2-70e3-4b51-a4ad-62630be33b67 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:44:41.980552 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-743456d2-70e3-4b51-a4ad-62630be33b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:44:42.164012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 2/3] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:44:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 518 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 14 11:44:42.599210 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869babe-6a3b-4dd9-b843-e917efab6c8d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:42.669730 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: INFO dbcounter [None req-a869babe-6a3b-4dd9-b843-e917efab6c8d None None] Registered counter for database keystone Sep 14 11:44:42.690991 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG oslo_db.sqlalchemy.engines [None req-a869babe-6a3b-4dd9-b843-e917efab6c8d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81582) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 14 11:44:42.697092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writer thread running {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 14 11:44:42.815460 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869babe-6a3b-4dd9-b843-e917efab6c8d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:42.819813 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe64637-00b8-49e3-8978-c65b153e2c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:42.820455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe64637-00b8-49e3-8978-c65b153e2c2f admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:42.820740 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe64637-00b8-49e3-8978-c65b153e2c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:42.821019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe64637-00b8-49e3-8978-c65b153e2c2f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:42.830079 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe64637-00b8-49e3-8978-c65b153e2c2f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:42.881590 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: WARNING keystone.common.rbac_enforcer.enforcer [None req-efe64637-00b8-49e3-8978-c65b153e2c2f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 14 11:44:42.881907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe64637-00b8-49e3-8978-c65b153e2c2f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:42.891872 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 2/4] 10.222.0.62 () {58 vars in 1324 bytes} [Wed Sep 14 11:44:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 11:44:42.897956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743456d2-70e3-4b51-a4ad-62630be33b67 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:42.914779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743456d2-70e3-4b51-a4ad-62630be33b67 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:42.919219 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aed997-7fec-425a-8363-fe1cb394e8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:42.920334 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aed997-7fec-425a-8363-fe1cb394e8cd admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:42.920620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aed997-7fec-425a-8363-fe1cb394e8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:42.920903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aed997-7fec-425a-8363-fe1cb394e8cd admin admin] PATH_INFO: `/v3/projects/a95d34dd28804fc3b45b9488a19047b4` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:42.922535 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aed997-7fec-425a-8363-fe1cb394e8cd admin admin] RBAC: Authorizing `identity:get_project(project_id=a95d34dd28804fc3b45b9488a19047b4)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:42.975814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: WARNING keystone.common.rbac_enforcer.enforcer [None req-16aed997-7fec-425a-8363-fe1cb394e8cd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 14 11:44:42.976164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aed997-7fec-425a-8363-fe1cb394e8cd admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:42.980660 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 3/5] 10.222.0.62 () {56 vars in 1207 bytes} [Wed Sep 14 11:44:42 2022] GET /identity/v3/projects/a95d34dd28804fc3b45b9488a19047b4 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:44:45.195711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd5811-ddd0-4783-b325-43b2a241c032 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:45.196231 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd5811-ddd0-4783-b325-43b2a241c032 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:45.196343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd5811-ddd0-4783-b325-43b2a241c032 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:45.200339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 3/6] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:44:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:44:45.205789 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a53fe2-7099-4927-bb4d-31e1fba115dc None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:45.206222 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a53fe2-7099-4927-bb4d-31e1fba115dc None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:45.206386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a53fe2-7099-4927-bb4d-31e1fba115dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:45.226680 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-40a53fe2-7099-4927-bb4d-31e1fba115dc None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:44:45.228224 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-40a53fe2-7099-4927-bb4d-31e1fba115dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:44:45.236782 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 4/7] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:44:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 14 11:44:45.505098 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd5811-ddd0-4783-b325-43b2a241c032 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:45.517579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd5811-ddd0-4783-b325-43b2a241c032 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:45.521774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6740eb-83de-4d8c-b935-f89ab92bf322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:45.522405 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6740eb-83de-4d8c-b935-f89ab92bf322 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:45.522688 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6740eb-83de-4d8c-b935-f89ab92bf322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:45.522962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6740eb-83de-4d8c-b935-f89ab92bf322 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:45.530494 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6740eb-83de-4d8c-b935-f89ab92bf322 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:45.532889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6740eb-83de-4d8c-b935-f89ab92bf322 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:45.541445 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 4/8] 10.222.0.62 () {58 vars in 1324 bytes} [Wed Sep 14 11:44:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 11:44:45.546636 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a53fe2-7099-4927-bb4d-31e1fba115dc None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:45.555932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a53fe2-7099-4927-bb4d-31e1fba115dc None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:45.560283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173630f-1408-4c4a-a155-37e0f76fd019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:45.560910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173630f-1408-4c4a-a155-37e0f76fd019 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:45.561200 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173630f-1408-4c4a-a155-37e0f76fd019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:45.561488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173630f-1408-4c4a-a155-37e0f76fd019 admin admin] PATH_INFO: `/v3/users/6c2df9ec255e48ec9c7b4291f711f0cd` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:45.562843 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173630f-1408-4c4a-a155-37e0f76fd019 admin admin] RBAC: Authorizing `identity:get_user(user_id=6c2df9ec255e48ec9c7b4291f711f0cd)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:45.564674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173630f-1408-4c4a-a155-37e0f76fd019 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:45.566242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 5/9] 10.222.0.62 () {56 vars in 1198 bytes} [Wed Sep 14 11:44:45 2022] GET /identity/v3/users/6c2df9ec255e48ec9c7b4291f711f0cd => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:44:47.891365 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6637e-5834-4574-baa9-e76fc20e7a64 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:47.891906 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6637e-5834-4574-baa9-e76fc20e7a64 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:47.891906 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6637e-5834-4574-baa9-e76fc20e7a64 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:47.893253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 5/10] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:44:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:44:47.898526 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08fc0e7-224a-4562-9c69-44da56ad2757 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:47.898921 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08fc0e7-224a-4562-9c69-44da56ad2757 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:47.899109 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08fc0e7-224a-4562-9c69-44da56ad2757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:47.918721 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d08fc0e7-224a-4562-9c69-44da56ad2757 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:44:47.920248 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d08fc0e7-224a-4562-9c69-44da56ad2757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:44:47.963537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd3cb5-e069-4969-987b-c2c88c90c3cb None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:47.963817 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd3cb5-e069-4969-987b-c2c88c90c3cb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:47.964096 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd3cb5-e069-4969-987b-c2c88c90c3cb None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:47.965404 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 6/11] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:44:47.971861 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0508c-20ef-4dff-8759-f030e91c0e5a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:47.972311 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0508c-20ef-4dff-8759-f030e91c0e5a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:47.972597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0508c-20ef-4dff-8759-f030e91c0e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:48.012160 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 6/12] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:44:48.025419 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG passlib.handlers.bcrypt [None req-eae0508c-20ef-4dff-8759-f030e91c0e5a None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81582) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 14 11:44:48.025815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG passlib.handlers.bcrypt [None req-eae0508c-20ef-4dff-8759-f030e91c0e5a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81582) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 14 11:44:48.079769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-eae0508c-20ef-4dff-8759-f030e91c0e5a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:44:48.080193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-eae0508c-20ef-4dff-8759-f030e91c0e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:44:48.193233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 7/13] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:44:48.296936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08fc0e7-224a-4562-9c69-44da56ad2757 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:48.327791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:48.414903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08fc0e7-224a-4562-9c69-44da56ad2757 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:48.449582 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe4b67-3a09-408d-bdfc-3e19bc538056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:48.450342 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe4b67-3a09-408d-bdfc-3e19bc538056 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:48.450634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe4b67-3a09-408d-bdfc-3e19bc538056 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:48.450917 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe4b67-3a09-408d-bdfc-3e19bc538056 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:48.462559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe4b67-3a09-408d-bdfc-3e19bc538056 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:44:48.471256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-dffe4b67-3a09-408d-bdfc-3e19bc538056 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:44:48.471256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:44:48.471256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:44:48.471256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:44:48.471256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:44:48.471256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:44:48.471256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:44:48.471256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:44:48.471256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:44:48.471256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:44:48.471256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:44:48.471256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:44:48.471256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:44:48.471256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:44:48.471256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:44:48.471256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:44:48.472464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:44:48.472464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:44:48.472464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:44:48.472464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:44:48.472464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:44:48.472464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:44:48.472464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:44:48.472464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:44:48.472464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:44:48.472464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:44:48.472464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:44:48.472464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:44:48.472464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:44:48.472464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:44:48.472464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:44:48.472464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:44:48.473373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:44:48.473373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:44:48.473373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:44:48.473373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:44:48.473373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:44:48.473373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:44:48.473373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:44:48.473373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:44:48.473373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:44:48.473373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:44:48.473373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:44:48.473373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:44:48.473373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 7/14] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:44:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:44:48.474235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0508c-20ef-4dff-8759-f030e91c0e5a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:48.480295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe4b67-3a09-408d-bdfc-3e19bc538056 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:48.563517 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:48.578031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0508c-20ef-4dff-8759-f030e91c0e5a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:48.580442 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:48.583095 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe4b67-3a09-408d-bdfc-3e19bc538056 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:48.617989 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851b315-b088-4e82-86e9-48ba0b53a8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:48.618853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851b315-b088-4e82-86e9-48ba0b53a8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:48.619136 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851b315-b088-4e82-86e9-48ba0b53a8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:48.619412 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851b315-b088-4e82-86e9-48ba0b53a8e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:48.619501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3563ab-d8f3-412a-8f49-bd4ef966b94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:48.620156 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3563ab-d8f3-412a-8f49-bd4ef966b94b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:48.620613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3563ab-d8f3-412a-8f49-bd4ef966b94b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:48.620908 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3563ab-d8f3-412a-8f49-bd4ef966b94b None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:48.621568 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3563ab-d8f3-412a-8f49-bd4ef966b94b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:44:48.621913 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3563ab-d8f3-412a-8f49-bd4ef966b94b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:48.623308 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3563ab-d8f3-412a-8f49-bd4ef966b94b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:48.636154 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 8/15] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:44:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:44:48.641974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3563ab-d8f3-412a-8f49-bd4ef966b94b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:48.690488 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851b315-b088-4e82-86e9-48ba0b53a8e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:48.691938 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851b315-b088-4e82-86e9-48ba0b53a8e2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:48.745117 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3563ab-d8f3-412a-8f49-bd4ef966b94b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:48.777839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60b7ca-78ac-4a44-991c-546e21b56ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:48.778490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60b7ca-78ac-4a44-991c-546e21b56ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:48.778781 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60b7ca-78ac-4a44-991c-546e21b56ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:48.779061 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60b7ca-78ac-4a44-991c-546e21b56ed5 None admin] PATH_INFO: `/v3/users/6c2df9ec255e48ec9c7b4291f711f0cd` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:48.780212 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60b7ca-78ac-4a44-991c-546e21b56ed5 None admin] RBAC: Authorizing `identity:get_user(user_id=6c2df9ec255e48ec9c7b4291f711f0cd)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:48.782244 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60b7ca-78ac-4a44-991c-546e21b56ed5 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:48.783795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 9/16] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:44:48 2022] GET /identity/v3/users/6c2df9ec255e48ec9c7b4291f711f0cd => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:44:48.789303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60b7ca-78ac-4a44-991c-546e21b56ed5 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:48.790863 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 8/17] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:44:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:44:48.796348 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851b315-b088-4e82-86e9-48ba0b53a8e2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:48.799655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:48.804848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:48.893131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60b7ca-78ac-4a44-991c-546e21b56ed5 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:48.902004 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851b315-b088-4e82-86e9-48ba0b53a8e2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:48.929804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e363673-9ea8-4b57-906f-f78404352e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:48.930491 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e363673-9ea8-4b57-906f-f78404352e58 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:48.931679 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e363673-9ea8-4b57-906f-f78404352e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:48.931679 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e363673-9ea8-4b57-906f-f78404352e58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:48.932306 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e363673-9ea8-4b57-906f-f78404352e58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:48.933827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e363673-9ea8-4b57-906f-f78404352e58 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:48.935525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 10/18] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:44:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:44:48.940460 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd82af-27ef-4c62-b8b5-ae9890145b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:48.941723 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd82af-27ef-4c62-b8b5-ae9890145b73 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:48.942055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd82af-27ef-4c62-b8b5-ae9890145b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:48.942365 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd82af-27ef-4c62-b8b5-ae9890145b73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:48.942610 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e363673-9ea8-4b57-906f-f78404352e58 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:48.961467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd82af-27ef-4c62-b8b5-ae9890145b73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:48.962969 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd82af-27ef-4c62-b8b5-ae9890145b73 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:48.982858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-e9fd82af-27ef-4c62-b8b5-ae9890145b73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:44:48.982858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:44:48.982858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:44:48.982858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:44:48.982858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:44:48.982858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:44:48.982858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:44:48.982858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:44:48.982858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:44:48.982858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:44:48.982858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:44:48.982858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:44:48.982858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:44:48.982858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:44:48.982858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:44:48.982858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:44:48.983835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:44:48.983835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:44:48.983835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:44:48.983835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:44:48.983835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:44:48.983835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:44:48.983835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:44:48.983835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:44:48.983835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:44:48.983835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:44:48.983835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:44:48.983835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:44:48.983835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:44:48.983835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:44:48.983835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:44:48.983835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:44:48.983835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:44:48.985267 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:44:48.985267 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:44:48.985267 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:44:48.985267 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 9/19] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:44:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:44:48.990532 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd82af-27ef-4c62-b8b5-ae9890145b73 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:49.027382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:49.046407 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e363673-9ea8-4b57-906f-f78404352e58 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:49.050892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:49.081712 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65b331-7b83-4f82-b492-f29d69dbf5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:49.082352 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65b331-7b83-4f82-b492-f29d69dbf5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:49.082641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65b331-7b83-4f82-b492-f29d69dbf5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:49.082921 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65b331-7b83-4f82-b492-f29d69dbf5c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:49.083651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65b331-7b83-4f82-b492-f29d69dbf5c3 None admin] RBAC: Adding query filter params (role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.domain.id=default, user.id=6c2df9ec255e48ec9c7b4291f711f0cd) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:44:49.083963 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65b331-7b83-4f82-b492-f29d69dbf5c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:49.085404 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65b331-7b83-4f82-b492-f29d69dbf5c3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:49.092001 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 11/20] 10.222.0.62 () {56 vars in 1322 bytes} [Wed Sep 14 11:44:48 2022] GET /identity/v3/role_assignments?user.id=6c2df9ec255e48ec9c7b4291f711f0cd&scope.domain.id=default&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:44:49.096422 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd82af-27ef-4c62-b8b5-ae9890145b73 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:49.134532 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3c67f-42bf-48fc-bb19-50e8d95d9b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:49.135156 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3c67f-42bf-48fc-bb19-50e8d95d9b06 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:49.135438 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3c67f-42bf-48fc-bb19-50e8d95d9b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:49.135869 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3c67f-42bf-48fc-bb19-50e8d95d9b06 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:49.136532 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d3c67f-42bf-48fc-bb19-50e8d95d9b06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:44:49.136830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d3c67f-42bf-48fc-bb19-50e8d95d9b06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:49.138296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d3c67f-42bf-48fc-bb19-50e8d95d9b06 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:49.158853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 10/21] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:44:51.558886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586071b8-e687-4ca0-a354-06a926320837 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:51.559925 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586071b8-e687-4ca0-a354-06a926320837 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:51.560416 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586071b8-e687-4ca0-a354-06a926320837 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:51.562012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 12/22] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:44:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:44:51.567560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68c26d-73ff-4ff7-bdad-2e499e978fa8 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:51.568004 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68c26d-73ff-4ff7-bdad-2e499e978fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:51.568177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68c26d-73ff-4ff7-bdad-2e499e978fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:51.596126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7a68c26d-73ff-4ff7-bdad-2e499e978fa8 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:44:51.597347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7a68c26d-73ff-4ff7-bdad-2e499e978fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:44:51.688846 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 11/23] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:44:51.932069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4c6c3-2f5d-4efa-a079-b8ea0f064d34 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:51.932728 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4c6c3-2f5d-4efa-a079-b8ea0f064d34 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:51.933221 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4c6c3-2f5d-4efa-a079-b8ea0f064d34 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:51.935077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 13/24] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:44:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:44:51.945363 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db5359-0b86-4459-b4e7-4347ab913448 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:51.945972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db5359-0b86-4459-b4e7-4347ab913448 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:51.946573 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db5359-0b86-4459-b4e7-4347ab913448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:51.972836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-76db5359-0b86-4459-b4e7-4347ab913448 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:44:51.974382 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-76db5359-0b86-4459-b4e7-4347ab913448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:44:52.021798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:52.059850 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 12/25] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:44:52.182184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2ecb9-7c66-4e40-8560-709439b88b97 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:52.182710 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2ecb9-7c66-4e40-8560-709439b88b97 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:52.183135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2ecb9-7c66-4e40-8560-709439b88b97 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:52.184942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 14/26] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:44:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:44:52.190045 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2455784-e899-452d-8e1b-3afdcc6f4afd None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:52.190343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2455784-e899-452d-8e1b-3afdcc6f4afd None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:52.190623 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2455784-e899-452d-8e1b-3afdcc6f4afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:52.193594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2ecb9-7c66-4e40-8560-709439b88b97 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:52.255408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a2455784-e899-452d-8e1b-3afdcc6f4afd None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:44:52.255408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a2455784-e899-452d-8e1b-3afdcc6f4afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:44:52.479020 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 13/27] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:44:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:44:52.480099 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2455784-e899-452d-8e1b-3afdcc6f4afd None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:52.517409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2ecb9-7c66-4e40-8560-709439b88b97 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:52.608275 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc0d90-ab5e-46b7-bce0-d7c9469101f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:52.609658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc0d90-ab5e-46b7-bce0-d7c9469101f4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:52.610114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc0d90-ab5e-46b7-bce0-d7c9469101f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:52.610552 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc0d90-ab5e-46b7-bce0-d7c9469101f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:52.644675 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc0d90-ab5e-46b7-bce0-d7c9469101f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:44:52.650720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-f5fc0d90-ab5e-46b7-bce0-d7c9469101f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:44:52.650720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:44:52.650720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:44:52.650720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:44:52.650720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:44:52.650720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:44:52.650720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:44:52.650720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:44:52.650720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:44:52.650720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:44:52.650720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:44:52.650720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:44:52.650720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:44:52.650720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:44:52.650720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:44:52.650720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:44:52.651576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:44:52.651576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:44:52.651576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:44:52.651576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:44:52.651576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:44:52.651576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:44:52.651576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:44:52.651576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:44:52.651576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:44:52.651576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:44:52.651576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:44:52.651576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:44:52.651576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:44:52.651576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:44:52.651576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:44:52.651576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:44:52.652339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:44:52.652339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:44:52.652339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:44:52.652339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:44:52.652339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:44:52.652339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:44:52.652339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:44:52.652339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:44:52.652339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:44:52.652339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:44:52.652339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:44:52.652339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:44:52.661217 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 15/28] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:44:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 468 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:44:52.675910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc0d90-ab5e-46b7-bce0-d7c9469101f4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:52.686203 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:52.801702 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2455784-e899-452d-8e1b-3afdcc6f4afd None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:52.819591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:52.918624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a1704-650c-4289-bb84-26e303f74435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:52.919273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a1704-650c-4289-bb84-26e303f74435 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:52.919563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a1704-650c-4289-bb84-26e303f74435 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:52.919843 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a1704-650c-4289-bb84-26e303f74435 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:52.920615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a1704-650c-4289-bb84-26e303f74435 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:52.922656 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a1704-650c-4289-bb84-26e303f74435 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:52.961248 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc0d90-ab5e-46b7-bce0-d7c9469101f4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:53.019474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2562a-8a51-44f3-a761-054677302752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:53.020121 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2562a-8a51-44f3-a761-054677302752 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:53.020444 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2562a-8a51-44f3-a761-054677302752 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:53.020914 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2562a-8a51-44f3-a761-054677302752 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:53.021620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2562a-8a51-44f3-a761-054677302752 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:44:53.021955 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2562a-8a51-44f3-a761-054677302752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:53.023405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2562a-8a51-44f3-a761-054677302752 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:53.032596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 16/29] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:44:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:44:53.033906 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fcd1e-f9a6-44f2-8ea7-9cbd7c4cb4eb None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:53.034210 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fcd1e-f9a6-44f2-8ea7-9cbd7c4cb4eb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:53.034493 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fcd1e-f9a6-44f2-8ea7-9cbd7c4cb4eb None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:53.035702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 17/30] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:44:53.040817 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fcd1e-f9a6-44f2-8ea7-9cbd7c4cb4eb None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:53.119827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:53.142163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 14/31] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Sep 14 11:44:52 2022] POST /identity/v3/roles => generated 219 bytes in 662 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:44:53.143239 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a1704-650c-4289-bb84-26e303f74435 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:53.170020 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fcd1e-f9a6-44f2-8ea7-9cbd7c4cb4eb None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:53.213433 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb44f53-d8f7-4bce-8362-447565be12d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:53.214571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb44f53-d8f7-4bce-8362-447565be12d1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:53.215085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb44f53-d8f7-4bce-8362-447565be12d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:53.215568 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb44f53-d8f7-4bce-8362-447565be12d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:53.229263 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb44f53-d8f7-4bce-8362-447565be12d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:53.231244 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb44f53-d8f7-4bce-8362-447565be12d1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:53.249395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-1fb44f53-d8f7-4bce-8362-447565be12d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:44:53.249395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:44:53.249395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:44:53.249395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:44:53.249395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:44:53.249395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:44:53.249395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:44:53.249395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:44:53.249395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:44:53.249395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:44:53.249395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:44:53.249395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:44:53.249395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:44:53.249395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:44:53.249395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:44:53.249395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:44:53.250394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:44:53.250394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:44:53.250394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:44:53.250394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:44:53.250394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:44:53.250394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:44:53.250394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:44:53.250394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:44:53.250394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:44:53.250394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:44:53.250394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:44:53.250394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:44:53.250394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:44:53.250394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:44:53.250394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:44:53.250394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:44:53.250394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:44:53.251205 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:44:53.251205 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:44:53.251205 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:44:53.253243 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 18/32] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:44:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:44:53.254883 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510664b2-83ce-40d3-b972-8c1011d3c5df None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:53.255464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510664b2-83ce-40d3-b972-8c1011d3c5df None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:53.255943 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510664b2-83ce-40d3-b972-8c1011d3c5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:53.276646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-510664b2-83ce-40d3-b972-8c1011d3c5df None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:44:53.278292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-510664b2-83ce-40d3-b972-8c1011d3c5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:44:53.284184 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a1704-650c-4289-bb84-26e303f74435 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:53.327785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d631c-ba8c-4af2-a060-2c0fe9bc8997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:53.328328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d631c-ba8c-4af2-a060-2c0fe9bc8997 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:53.328621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d631c-ba8c-4af2-a060-2c0fe9bc8997 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:53.328906 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d631c-ba8c-4af2-a060-2c0fe9bc8997 None admin] PATH_INFO: `/v3/users/6c2df9ec255e48ec9c7b4291f711f0cd` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:53.330724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d631c-ba8c-4af2-a060-2c0fe9bc8997 None admin] RBAC: Authorizing `identity:get_user(user_id=6c2df9ec255e48ec9c7b4291f711f0cd)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:53.332410 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d631c-ba8c-4af2-a060-2c0fe9bc8997 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:53.334046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 15/33] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:44:53 2022] GET /identity/v3/users/6c2df9ec255e48ec9c7b4291f711f0cd => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:44:53.336097 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86720114-d660-45f2-9ce9-c0085bccf866 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:53.336390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86720114-d660-45f2-9ce9-c0085bccf866 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:53.336953 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86720114-d660-45f2-9ce9-c0085bccf866 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:53.338744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 16/34] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:44:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:44:53.339640 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86720114-d660-45f2-9ce9-c0085bccf866 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:53.361329 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 19/35] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:44:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:44:53.362355 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510664b2-83ce-40d3-b972-8c1011d3c5df None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:53.401096 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:53.438896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:53.483660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86720114-d660-45f2-9ce9-c0085bccf866 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:53.497632 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510664b2-83ce-40d3-b972-8c1011d3c5df None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:53.523569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc06e4-3bb6-4477-8ec9-49b925ab1be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:53.524523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc06e4-3bb6-4477-8ec9-49b925ab1be2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:53.524964 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc06e4-3bb6-4477-8ec9-49b925ab1be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:53.525241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc06e4-3bb6-4477-8ec9-49b925ab1be2 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:53.526000 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc06e4-3bb6-4477-8ec9-49b925ab1be2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:44:53.526316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc06e4-3bb6-4477-8ec9-49b925ab1be2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:53.527712 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc06e4-3bb6-4477-8ec9-49b925ab1be2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:53.535742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4dd8d3-e2f7-4489-8e87-e074e82b288b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:53.536238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dd8d3-e2f7-4489-8e87-e074e82b288b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:53.536519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dd8d3-e2f7-4489-8e87-e074e82b288b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:53.536792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dd8d3-e2f7-4489-8e87-e074e82b288b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:53.537938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4dd8d3-e2f7-4489-8e87-e074e82b288b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:53.539376 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4dd8d3-e2f7-4489-8e87-e074e82b288b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:53.545802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 20/37] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:44:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:44:53.545964 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 17/37] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:44:53.546072 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df23fa-ca27-4462-9973-0d6eaa2d619a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:53.546154 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628658f-8909-4fb8-bc5b-fda4183ca5df None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:53.546230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df23fa-ca27-4462-9973-0d6eaa2d619a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:53.546297 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628658f-8909-4fb8-bc5b-fda4183ca5df None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:53.546362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df23fa-ca27-4462-9973-0d6eaa2d619a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:53.546436 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628658f-8909-4fb8-bc5b-fda4183ca5df None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:53.546436 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 18/38] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:44:53.546436 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628658f-8909-4fb8-bc5b-fda4183ca5df None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:53.567509 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-25df23fa-ca27-4462-9973-0d6eaa2d619a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:44:53.568585 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-25df23fa-ca27-4462-9973-0d6eaa2d619a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:44:53.639262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 21/39] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:44:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:44:53.641035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a439ce-6e3b-4f81-876e-2d9af0397032 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:53.641331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a439ce-6e3b-4f81-876e-2d9af0397032 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:53.641613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a439ce-6e3b-4f81-876e-2d9af0397032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:53.668416 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:53.668652 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a7a439ce-6e3b-4f81-876e-2d9af0397032 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:44:53.669895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a7a439ce-6e3b-4f81-876e-2d9af0397032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:44:53.676965 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628658f-8909-4fb8-bc5b-fda4183ca5df None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:53.712707 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:53.713813 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19b1a8-0535-4478-b3af-62eacde1ea22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:53.714786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19b1a8-0535-4478-b3af-62eacde1ea22 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:53.715239 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19b1a8-0535-4478-b3af-62eacde1ea22 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:53.715703 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19b1a8-0535-4478-b3af-62eacde1ea22 None admin] PATH_INFO: `/v3/domains/default/users/6c2df9ec255e48ec9c7b4291f711f0cd/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:53.718270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19b1a8-0535-4478-b3af-62eacde1ea22 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6c2df9ec255e48ec9c7b4291f711f0cd, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:53.720860 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19b1a8-0535-4478-b3af-62eacde1ea22 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:53.738452 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 22/40] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:44:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:44:53.739269 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a439ce-6e3b-4f81-876e-2d9af0397032 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:53.846490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 19/41] 10.222.0.62 () {58 vars in 1361 bytes} [Wed Sep 14 11:44:53 2022] PUT /identity/v3/domains/default/users/6c2df9ec255e48ec9c7b4291f711f0cd/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:44:53.847002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a439ce-6e3b-4f81-876e-2d9af0397032 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:53.848424 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19b1a8-0535-4478-b3af-62eacde1ea22 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:53.949255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2382f-5be4-4b4d-80bd-ebaf4e5461c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:53.949809 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2382f-5be4-4b4d-80bd-ebaf4e5461c3 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:53.950170 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2382f-5be4-4b4d-80bd-ebaf4e5461c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:53.950456 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2382f-5be4-4b4d-80bd-ebaf4e5461c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:53.951188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2382f-5be4-4b4d-80bd-ebaf4e5461c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:53.952695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2382f-5be4-4b4d-80bd-ebaf4e5461c3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:53.995036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19b1a8-0535-4478-b3af-62eacde1ea22 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:54.035133 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f300ea3-c9c5-4e8c-8feb-f8b02fd26148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:54.035793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f300ea3-c9c5-4e8c-8feb-f8b02fd26148 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:54.036090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f300ea3-c9c5-4e8c-8feb-f8b02fd26148 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:54.036376 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f300ea3-c9c5-4e8c-8feb-f8b02fd26148 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:54.037174 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f300ea3-c9c5-4e8c-8feb-f8b02fd26148 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:54.038766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f300ea3-c9c5-4e8c-8feb-f8b02fd26148 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:54.153221 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 20/42] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Sep 14 11:44:53 2022] POST /identity/v3/roles => generated 225 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:44:54.154108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f300ea3-c9c5-4e8c-8feb-f8b02fd26148 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:54.178590 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:54.181498 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 23/43] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 14 11:44:53 2022] POST /identity/v3/projects => generated 301 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:44:54.182949 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2382f-5be4-4b4d-80bd-ebaf4e5461c3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:54.223920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:54.395304 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f300ea3-c9c5-4e8c-8feb-f8b02fd26148 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:54.447448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c20e2-6aa0-4d56-807b-10c9c23afff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:54.447448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c20e2-6aa0-4d56-807b-10c9c23afff1 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:54.447730 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c20e2-6aa0-4d56-807b-10c9c23afff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:54.448024 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c20e2-6aa0-4d56-807b-10c9c23afff1 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:54.448717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c20e2-6aa0-4d56-807b-10c9c23afff1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:54.450168 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c20e2-6aa0-4d56-807b-10c9c23afff1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:54.450834 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2382f-5be4-4b4d-80bd-ebaf4e5461c3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:54.494673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa03f6-5a74-4b18-a752-74fc73313cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:54.495745 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa03f6-5a74-4b18-a752-74fc73313cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:54.496312 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa03f6-5a74-4b18-a752-74fc73313cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:54.496754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa03f6-5a74-4b18-a752-74fc73313cf6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:54.498047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa03f6-5a74-4b18-a752-74fc73313cf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:54.500311 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa03f6-5a74-4b18-a752-74fc73313cf6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:54.502014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 24/44] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:44:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:44:54.507560 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa03f6-5a74-4b18-a752-74fc73313cf6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:54.525260 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 21/45] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Sep 14 11:44:54 2022] POST /identity/v3/roles => generated 223 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:44:54.619700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:54.627425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa03f6-5a74-4b18-a752-74fc73313cf6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:54.666093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc004cb-da1c-4467-a9c4-6b9a49d952eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:54.666735 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc004cb-da1c-4467-a9c4-6b9a49d952eb None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:54.667010 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc004cb-da1c-4467-a9c4-6b9a49d952eb None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:54.667295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc004cb-da1c-4467-a9c4-6b9a49d952eb None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:54.667995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc004cb-da1c-4467-a9c4-6b9a49d952eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:54.669508 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc004cb-da1c-4467-a9c4-6b9a49d952eb None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:54.755231 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 25/46] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 14 11:44:54 2022] POST /identity/v3/projects => generated 312 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 11:44:56.131923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9a23a-d866-41d8-a6a5-3c46a9c32dc4 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:56.132436 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9a23a-d866-41d8-a6a5-3c46a9c32dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:56.132519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9a23a-d866-41d8-a6a5-3c46a9c32dc4 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:56.133833 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 22/47] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:44:56.139208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35700ce1-5d8a-4aad-b54b-1c283a2fd176 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:56.139597 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35700ce1-5d8a-4aad-b54b-1c283a2fd176 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:56.139787 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35700ce1-5d8a-4aad-b54b-1c283a2fd176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:56.159041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-35700ce1-5d8a-4aad-b54b-1c283a2fd176 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:44:56.160590 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-35700ce1-5d8a-4aad-b54b-1c283a2fd176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:44:56.225850 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 26/48] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:44:56.492194 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9a23a-d866-41d8-a6a5-3c46a9c32dc4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:56.568109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:56.593193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9a23a-d866-41d8-a6a5-3c46a9c32dc4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:56.627185 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50764874-e18f-48bc-a6ac-cbf6352857fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:56.627820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50764874-e18f-48bc-a6ac-cbf6352857fd None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:56.628107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50764874-e18f-48bc-a6ac-cbf6352857fd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:56.628387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50764874-e18f-48bc-a6ac-cbf6352857fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:56.640502 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50764874-e18f-48bc-a6ac-cbf6352857fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:44:56.646596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-50764874-e18f-48bc-a6ac-cbf6352857fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:44:56.646596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:44:56.646596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:44:56.646596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:44:56.646596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:44:56.646596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:44:56.646596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:44:56.646596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:44:56.646596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:44:56.646596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:44:56.646596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:44:56.646596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:44:56.646596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:44:56.646596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:44:56.646596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:44:56.646596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:44:56.647439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:44:56.647439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:44:56.647439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:44:56.647439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:44:56.647439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:44:56.647439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:44:56.647439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:44:56.647439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:44:56.647439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:44:56.647439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:44:56.647439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:44:56.647439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:44:56.647439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:44:56.647439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:44:56.647439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:44:56.647439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:44:56.648215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:44:56.648215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:44:56.648215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:44:56.648215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:44:56.648215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:44:56.648215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:44:56.648215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:44:56.648215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:44:56.648215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:44:56.648215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:44:56.648215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:44:56.648215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:44:56.648215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 23/49] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:44:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:44:56.653617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35700ce1-5d8a-4aad-b54b-1c283a2fd176 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:56.730643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:56.752749 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35700ce1-5d8a-4aad-b54b-1c283a2fd176 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:56.789615 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1f046-8c44-42fd-be17-b8a78b233912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:56.790280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1f046-8c44-42fd-be17-b8a78b233912 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:56.790573 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1f046-8c44-42fd-be17-b8a78b233912 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:56.790857 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1f046-8c44-42fd-be17-b8a78b233912 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:56.791732 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1f046-8c44-42fd-be17-b8a78b233912 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:44:56.792046 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1f046-8c44-42fd-be17-b8a78b233912 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:56.793501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1f046-8c44-42fd-be17-b8a78b233912 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:56.803694 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 27/50] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:44:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:44:56.809143 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50764874-e18f-48bc-a6ac-cbf6352857fd None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:56.885169 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ad857-1d54-4039-87f6-3c544f50a328 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:56.885463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ad857-1d54-4039-87f6-3c544f50a328 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:56.885752 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ad857-1d54-4039-87f6-3c544f50a328 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:56.887188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 28/51] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:44:56.892141 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d8ce7-b578-4a28-a4e6-9072eb27fdb1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:56.892499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d8ce7-b578-4a28-a4e6-9072eb27fdb1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:56.892848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d8ce7-b578-4a28-a4e6-9072eb27fdb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:56.910987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-315d8ce7-b578-4a28-a4e6-9072eb27fdb1 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:44:56.911767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50764874-e18f-48bc-a6ac-cbf6352857fd None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:56.912185 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-315d8ce7-b578-4a28-a4e6-9072eb27fdb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:44:56.945932 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d9f28-7fc0-4554-a824-b9140fa043ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:56.946561 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d9f28-7fc0-4554-a824-b9140fa043ce None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:56.946852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d9f28-7fc0-4554-a824-b9140fa043ce None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:56.947137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d9f28-7fc0-4554-a824-b9140fa043ce None admin] PATH_INFO: `/v3/users/6c2df9ec255e48ec9c7b4291f711f0cd` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:56.948279 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d9f28-7fc0-4554-a824-b9140fa043ce None admin] RBAC: Authorizing `identity:get_user(user_id=6c2df9ec255e48ec9c7b4291f711f0cd)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:56.949744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d9f28-7fc0-4554-a824-b9140fa043ce None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:56.951357 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 24/52] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:44:56 2022] GET /identity/v3/users/6c2df9ec255e48ec9c7b4291f711f0cd => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:44:56.956834 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d9f28-7fc0-4554-a824-b9140fa043ce None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:56.960340 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:56.978988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 29/53] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:44:57.055316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d9f28-7fc0-4554-a824-b9140fa043ce None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:57.087513 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6691cf2-1635-4836-a7e5-0e71fb592e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:57.088144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6691cf2-1635-4836-a7e5-0e71fb592e09 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:57.088580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6691cf2-1635-4836-a7e5-0e71fb592e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:57.088742 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6691cf2-1635-4836-a7e5-0e71fb592e09 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:57.089869 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6691cf2-1635-4836-a7e5-0e71fb592e09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:57.091311 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6691cf2-1635-4836-a7e5-0e71fb592e09 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:57.092863 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 25/54] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:44:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:44:57.099160 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d8ce7-b578-4a28-a4e6-9072eb27fdb1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:57.165608 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:57.197114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d8ce7-b578-4a28-a4e6-9072eb27fdb1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:57.231517 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39844826-0b3b-4058-8a5f-fba9dce431f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:57.232205 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39844826-0b3b-4058-8a5f-fba9dce431f0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:57.232560 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39844826-0b3b-4058-8a5f-fba9dce431f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:57.232884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39844826-0b3b-4058-8a5f-fba9dce431f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:57.233622 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39844826-0b3b-4058-8a5f-fba9dce431f0 None admin] RBAC: Adding query filter params (role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.domain.id=default, user.id=6c2df9ec255e48ec9c7b4291f711f0cd) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:44:57.234570 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39844826-0b3b-4058-8a5f-fba9dce431f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:57.236032 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39844826-0b3b-4058-8a5f-fba9dce431f0 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:57.241954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 30/55] 10.222.0.62 () {56 vars in 1322 bytes} [Wed Sep 14 11:44:57 2022] GET /identity/v3/role_assignments?user.id=6c2df9ec255e48ec9c7b4291f711f0cd&scope.domain.id=default&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:44:57.248670 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6691cf2-1635-4836-a7e5-0e71fb592e09 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:57.323644 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:57.461605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6691cf2-1635-4836-a7e5-0e71fb592e09 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:57.496187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6f05f-ffef-4fe0-8c21-28aab8a41254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:57.496820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6f05f-ffef-4fe0-8c21-28aab8a41254 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:57.497107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6f05f-ffef-4fe0-8c21-28aab8a41254 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:57.497390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6f05f-ffef-4fe0-8c21-28aab8a41254 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:57.498500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6f05f-ffef-4fe0-8c21-28aab8a41254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:57.499958 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6f05f-ffef-4fe0-8c21-28aab8a41254 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:57.501504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 26/56] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:44:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:44:57.507156 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39844826-0b3b-4058-8a5f-fba9dce431f0 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:44:57.605169 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39844826-0b3b-4058-8a5f-fba9dce431f0 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:44:57.636988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4f3ae-cc96-4acd-9dea-d18f8bc546cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:44:57.637589 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4f3ae-cc96-4acd-9dea-d18f8bc546cd None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:57.637901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4f3ae-cc96-4acd-9dea-d18f8bc546cd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:57.638182 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4f3ae-cc96-4acd-9dea-d18f8bc546cd None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:57.638826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4f3ae-cc96-4acd-9dea-d18f8bc546cd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:44:57.640796 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4f3ae-cc96-4acd-9dea-d18f8bc546cd None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:44:57.645182 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:44:57.730287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 31/57] 10.222.0.62 () {60 vars in 1140 bytes} [Wed Sep 14 11:44:57 2022] POST /identity/v3/projects => generated 298 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:44:59.839656 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a99cf-9479-4f59-bb10-f5e126f65630 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:59.840096 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a99cf-9479-4f59-bb10-f5e126f65630 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:59.840227 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a99cf-9479-4f59-bb10-f5e126f65630 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:59.841576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 27/58] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:44:59.847459 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cdad7-e657-46ac-90c3-948fa73d265a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:44:59.847857 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cdad7-e657-46ac-90c3-948fa73d265a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:44:59.848024 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cdad7-e657-46ac-90c3-948fa73d265a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:44:59.865668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-859cdad7-e657-46ac-90c3-948fa73d265a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:44:59.866901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-859cdad7-e657-46ac-90c3-948fa73d265a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:44:59.930603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 32/59] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:44:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:00.200306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a99cf-9479-4f59-bb10-f5e126f65630 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:00.299013 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a99cf-9479-4f59-bb10-f5e126f65630 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:00.332477 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e562c1-a5b0-459a-bbca-9456ec4552b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:00.333102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e562c1-a5b0-459a-bbca-9456ec4552b5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:00.333391 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e562c1-a5b0-459a-bbca-9456ec4552b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:00.333674 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e562c1-a5b0-459a-bbca-9456ec4552b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:00.334817 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e562c1-a5b0-459a-bbca-9456ec4552b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:00.336293 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e562c1-a5b0-459a-bbca-9456ec4552b5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:00.337935 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 28/60] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:45:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:00.343243 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cdad7-e657-46ac-90c3-948fa73d265a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:00.438845 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cdad7-e657-46ac-90c3-948fa73d265a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:00.454420 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:00.471136 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b7e84-6200-4af4-9166-bc07c19bec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:00.471771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b7e84-6200-4af4-9166-bc07c19bec10 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:00.472052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b7e84-6200-4af4-9166-bc07c19bec10 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:00.472331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b7e84-6200-4af4-9166-bc07c19bec10 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:00.472982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b7e84-6200-4af4-9166-bc07c19bec10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:00.474473 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b7e84-6200-4af4-9166-bc07c19bec10 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:00.546170 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 33/61] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 14 11:45:00 2022] POST /identity/v3/users => generated 275 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:45:03.873436 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2136329-0c73-4455-849c-4cbe443b1adc None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:03.874475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2136329-0c73-4455-849c-4cbe443b1adc None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:03.874960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2136329-0c73-4455-849c-4cbe443b1adc None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:03.876521 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 29/62] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:03.881853 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0827d23-898a-43cc-838a-66ad82b4bbcf None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:03.882283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0827d23-898a-43cc-838a-66ad82b4bbcf None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:03.882444 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0827d23-898a-43cc-838a-66ad82b4bbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:03.910488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d0827d23-898a-43cc-838a-66ad82b4bbcf None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:03.911984 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d0827d23-898a-43cc-838a-66ad82b4bbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:04.000891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 34/63] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:04.041422 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe718e08-59f5-4e2d-ad68-de86d0df2db9 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:04.041974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe718e08-59f5-4e2d-ad68-de86d0df2db9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:04.042470 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe718e08-59f5-4e2d-ad68-de86d0df2db9 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:04.044007 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 30/64] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:04.049904 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a03aa-57ee-40ae-80a6-8a451ba82b46 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:04.050699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a03aa-57ee-40ae-80a6-8a451ba82b46 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:04.051184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a03aa-57ee-40ae-80a6-8a451ba82b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:04.076879 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b17a03aa-57ee-40ae-80a6-8a451ba82b46 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:04.078947 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b17a03aa-57ee-40ae-80a6-8a451ba82b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:04.176857 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 35/65] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:04.297527 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe718e08-59f5-4e2d-ad68-de86d0df2db9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:04.306303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bbf72-a7be-43ba-8850-5ee901855b1c None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:04.306634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bbf72-a7be-43ba-8850-5ee901855b1c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:04.306878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bbf72-a7be-43ba-8850-5ee901855b1c None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:04.308583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 36/66] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:04.309075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:04.314213 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198c839-06f6-4099-956e-c0d651c74f9a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:04.314506 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198c839-06f6-4099-956e-c0d651c74f9a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:04.314792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198c839-06f6-4099-956e-c0d651c74f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:04.333754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-0198c839-06f6-4099-956e-c0d651c74f9a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:04.335039 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0198c839-06f6-4099-956e-c0d651c74f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:04.366223 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:04.423762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 37/67] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:45:04.425119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cdca7-079e-494e-8016-df38610fbd97 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:04.425681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cdca7-079e-494e-8016-df38610fbd97 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:04.426005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cdca7-079e-494e-8016-df38610fbd97 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:04.427410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 38/68] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:04.428671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6617d2-9d88-452f-9ae8-4b3003e0de94 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:04.428671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6617d2-9d88-452f-9ae8-4b3003e0de94 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:04.429030 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6617d2-9d88-452f-9ae8-4b3003e0de94 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:04.430393 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 39/69] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:04.435457 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67089a04-fa80-45b6-a0b6-acbf09a48778 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:04.435773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67089a04-fa80-45b6-a0b6-acbf09a48778 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:04.436012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67089a04-fa80-45b6-a0b6-acbf09a48778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:04.454477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-67089a04-fa80-45b6-a0b6-acbf09a48778 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:04.455697 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-67089a04-fa80-45b6-a0b6-acbf09a48778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:04.458003 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe718e08-59f5-4e2d-ad68-de86d0df2db9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:04.501921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327e825-3a2a-4511-9199-856a58932a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:04.502917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327e825-3a2a-4511-9199-856a58932a6c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:04.503765 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327e825-3a2a-4511-9199-856a58932a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:04.504249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327e825-3a2a-4511-9199-856a58932a6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:04.517452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327e825-3a2a-4511-9199-856a58932a6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:04.524484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-c327e825-3a2a-4511-9199-856a58932a6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:04.524484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:04.524484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:04.524484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:04.524484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:04.524484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:04.524484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:04.524484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:04.524484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:04.524484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:04.524484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:04.524484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:04.524484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:04.524484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:04.524484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:04.524484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:04.525326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:04.525326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:04.525326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:04.525326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:04.525326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:04.525326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:04.525326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:04.525326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:04.525326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:04.525326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:04.525326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:04.525326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:04.525326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:04.525326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:04.525326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:04.525326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:04.526130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:04.526130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:04.526130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:04.526130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:04.526130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:04.526130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:04.526130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:04.526130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:04.526130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:04.526130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:04.526130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:04.526130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:04.527307 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 31/70] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:04.529629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f3cfa-7f67-40b8-bd27-14774c9d4bff None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:04.529946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f3cfa-7f67-40b8-bd27-14774c9d4bff None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:04.530233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f3cfa-7f67-40b8-bd27-14774c9d4bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:04.535150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 40/71] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:04.536282 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67089a04-fa80-45b6-a0b6-acbf09a48778 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:04.549079 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0f8f3cfa-7f67-40b8-bd27-14774c9d4bff None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:04.550910 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0f8f3cfa-7f67-40b8-bd27-14774c9d4bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:04.614241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:04.623348 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 32/72] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:04.629504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f3cfa-7f67-40b8-bd27-14774c9d4bff None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:04.634702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:04.644509 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67089a04-fa80-45b6-a0b6-acbf09a48778 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:04.690215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7f096-c1ca-4842-adbc-724792fe2a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:04.690838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7f096-c1ca-4842-adbc-724792fe2a5e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:04.691163 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7f096-c1ca-4842-adbc-724792fe2a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:04.691398 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7f096-c1ca-4842-adbc-724792fe2a5e None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:04.692076 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7f096-c1ca-4842-adbc-724792fe2a5e None admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:04.692385 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7f096-c1ca-4842-adbc-724792fe2a5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:04.693787 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7f096-c1ca-4842-adbc-724792fe2a5e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:04.703616 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 41/73] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:04.716599 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7f096-c1ca-4842-adbc-724792fe2a5e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:04.791194 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f3cfa-7f67-40b8-bd27-14774c9d4bff None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:04.840255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf11cd-c079-4ff7-9baa-ff9b448dbdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:04.840899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf11cd-c079-4ff7-9baa-ff9b448dbdb6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:04.841185 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf11cd-c079-4ff7-9baa-ff9b448dbdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:04.841461 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf11cd-c079-4ff7-9baa-ff9b448dbdb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:04.841795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7f096-c1ca-4842-adbc-724792fe2a5e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:04.854146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf11cd-c079-4ff7-9baa-ff9b448dbdb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:04.860388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-94bf11cd-c079-4ff7-9baa-ff9b448dbdb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:45:04.860388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:04.860388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:04.860388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:04.860388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:04.860388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:04.860388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:04.860388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:04.860388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:04.860388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:04.860388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:04.860388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:04.860388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:04.860388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:04.860388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:04.860388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:04.861220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:04.861220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:04.861220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:04.861220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:04.861220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:04.861220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:04.861220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:04.861220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:04.861220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:04.861220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:04.861220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:04.861220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:04.861220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:04.861220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:04.861220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:04.861220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:04.863002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:04.863002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:04.863002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:04.863002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:04.863002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:04.863002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:04.863002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:04.863002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:04.863002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:04.863002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:04.863002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:45:04.863002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:04.863002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 33/74] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:45:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:04.863002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf11cd-c079-4ff7-9baa-ff9b448dbdb6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:04.879750 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9b2e9-14d1-4efd-961e-f6fd4f57a735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:04.880325 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9b2e9-14d1-4efd-961e-f6fd4f57a735 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:04.880615 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9b2e9-14d1-4efd-961e-f6fd4f57a735 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:04.880896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9b2e9-14d1-4efd-961e-f6fd4f57a735 None admin] PATH_INFO: `/v3/users/d56b1b366e684a00aa34e438baa482b8` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:04.895667 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9b2e9-14d1-4efd-961e-f6fd4f57a735 None admin] RBAC: Authorizing `identity:get_user(user_id=d56b1b366e684a00aa34e438baa482b8)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:04.897255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9b2e9-14d1-4efd-961e-f6fd4f57a735 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:04.899178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 42/75] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:04 2022] GET /identity/v3/users/d56b1b366e684a00aa34e438baa482b8 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:04.899807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9b2e9-14d1-4efd-961e-f6fd4f57a735 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:04.914813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:04.925691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:04.970389 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf11cd-c079-4ff7-9baa-ff9b448dbdb6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:05.004757 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9b2e9-14d1-4efd-961e-f6fd4f57a735 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:05.006300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a708b0-0794-4ef7-9997-9fd23c46252c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:05.006936 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a708b0-0794-4ef7-9997-9fd23c46252c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:05.007222 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a708b0-0794-4ef7-9997-9fd23c46252c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:05.007500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a708b0-0794-4ef7-9997-9fd23c46252c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:05.017787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a708b0-0794-4ef7-9997-9fd23c46252c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:05.022739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-67a708b0-0794-4ef7-9997-9fd23c46252c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:05.022739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:05.022739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:05.022739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:05.022739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:05.022739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:05.022739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:05.022739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:05.022739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:05.022739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:05.022739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:05.022739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:05.022739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:05.022739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:05.022739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:05.022739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:05.023762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:05.023762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:05.023762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:05.023762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:05.023762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:05.023762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:05.023762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:05.023762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:05.023762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:05.023762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:05.023762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:05.023762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:05.023762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:05.023762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:05.023762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:05.023762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:05.024546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:05.024546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:05.024546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:05.024546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:05.024546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:05.024546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:05.024546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:05.024546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:05.024546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:05.024546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:05.024546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:05.024546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:05.024546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 34/76] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:05.025254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a708b0-0794-4ef7-9997-9fd23c46252c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:05.039092 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efd1cd-a61e-4659-8688-24112c8a9de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:05.039738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efd1cd-a61e-4659-8688-24112c8a9de5 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:05.040055 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efd1cd-a61e-4659-8688-24112c8a9de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:05.040401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efd1cd-a61e-4659-8688-24112c8a9de5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:05.051525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90efd1cd-a61e-4659-8688-24112c8a9de5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:05.056472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-90efd1cd-a61e-4659-8688-24112c8a9de5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:05.056472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:05.056472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:05.056472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:05.056472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:05.056472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:05.056472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:05.056472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:05.056472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:05.056472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:05.056472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:05.056472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:05.056472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:05.056472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:05.056472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:05.056472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:05.057301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:05.057301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:05.057301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:05.057301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:05.057301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:05.057301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:05.057301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:05.057301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:05.057301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:05.057301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:05.057301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:05.057301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:05.057301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:05.057301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:05.057301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:05.057301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:05.058085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:05.058085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:05.058085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:05.058085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:05.058085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:05.058085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:05.058085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:05.058085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:05.058085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:05.058085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:05.058085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:05.058085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:05.058085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 43/77] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:45:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:05.058731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efd1cd-a61e-4659-8688-24112c8a9de5 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:05.128306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a708b0-0794-4ef7-9997-9fd23c46252c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:05.129567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:05.145139 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:05.157478 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efd1cd-a61e-4659-8688-24112c8a9de5 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:05.161806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2dfac-2804-4a67-9171-4d22e231fd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:05.162435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2dfac-2804-4a67-9171-4d22e231fd41 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:05.162737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2dfac-2804-4a67-9171-4d22e231fd41 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:05.163078 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2dfac-2804-4a67-9171-4d22e231fd41 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:05.163751 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2dfac-2804-4a67-9171-4d22e231fd41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:05.164060 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2dfac-2804-4a67-9171-4d22e231fd41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:05.165451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2dfac-2804-4a67-9171-4d22e231fd41 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:05.177919 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 35/78] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:05.178729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2dfac-2804-4a67-9171-4d22e231fd41 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:05.192697 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64bf729-2a2d-4153-8538-6406c2a31b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:05.193291 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bf729-2a2d-4153-8538-6406c2a31b9a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:05.193577 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bf729-2a2d-4153-8538-6406c2a31b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:05.193851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bf729-2a2d-4153-8538-6406c2a31b9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:05.260080 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64bf729-2a2d-4153-8538-6406c2a31b9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:05.261555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64bf729-2a2d-4153-8538-6406c2a31b9a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:05.276833 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2dfac-2804-4a67-9171-4d22e231fd41 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:05.311883 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d59516-7b54-4f42-b8d9-4bf36f009773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:05.312500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d59516-7b54-4f42-b8d9-4bf36f009773 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:05.312793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d59516-7b54-4f42-b8d9-4bf36f009773 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:05.313067 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d59516-7b54-4f42-b8d9-4bf36f009773 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:05.314209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d59516-7b54-4f42-b8d9-4bf36f009773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:05.315644 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d59516-7b54-4f42-b8d9-4bf36f009773 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:05.317183 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 36/79] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:45:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:05.317959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d59516-7b54-4f42-b8d9-4bf36f009773 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:05.334888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:05.364480 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:05.365518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 44/80] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:05.366453 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64bf729-2a2d-4153-8538-6406c2a31b9a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:05.422239 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d59516-7b54-4f42-b8d9-4bf36f009773 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:05.454684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de119306-368a-4eee-b478-f448fe716e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:05.455285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de119306-368a-4eee-b478-f448fe716e85 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:05.455567 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de119306-368a-4eee-b478-f448fe716e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:05.455842 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de119306-368a-4eee-b478-f448fe716e85 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:05.456494 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de119306-368a-4eee-b478-f448fe716e85 None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:05.456795 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de119306-368a-4eee-b478-f448fe716e85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:05.458233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de119306-368a-4eee-b478-f448fe716e85 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:05.465961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64bf729-2a2d-4153-8538-6406c2a31b9a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:05.466136 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 37/81] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:05.466927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de119306-368a-4eee-b478-f448fe716e85 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:05.504270 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f52672-8686-4abe-a7d3-3cab3318e2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:05.504881 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f52672-8686-4abe-a7d3-3cab3318e2de None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:05.505169 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f52672-8686-4abe-a7d3-3cab3318e2de None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:05.505446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f52672-8686-4abe-a7d3-3cab3318e2de None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:05.506146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f52672-8686-4abe-a7d3-3cab3318e2de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:05.506456 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f52672-8686-4abe-a7d3-3cab3318e2de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:05.507850 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f52672-8686-4abe-a7d3-3cab3318e2de None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:05.515666 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 45/82] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:45:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:05.521041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f52672-8686-4abe-a7d3-3cab3318e2de None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:05.556145 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:05.567712 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de119306-368a-4eee-b478-f448fe716e85 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:05.577170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:05.602275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb2e32-5e65-4d00-ace4-87f8afc4e07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:05.602924 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb2e32-5e65-4d00-ace4-87f8afc4e07f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:05.603208 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb2e32-5e65-4d00-ace4-87f8afc4e07f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:05.603488 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb2e32-5e65-4d00-ace4-87f8afc4e07f None admin] PATH_INFO: `/v3/users/6c2df9ec255e48ec9c7b4291f711f0cd` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:05.604642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb2e32-5e65-4d00-ace4-87f8afc4e07f None admin] RBAC: Authorizing `identity:get_user(user_id=6c2df9ec255e48ec9c7b4291f711f0cd)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:05.606149 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb2e32-5e65-4d00-ace4-87f8afc4e07f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:05.607725 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 38/83] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:05 2022] GET /identity/v3/users/6c2df9ec255e48ec9c7b4291f711f0cd => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:05.608540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb2e32-5e65-4d00-ace4-87f8afc4e07f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:05.621018 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f52672-8686-4abe-a7d3-3cab3318e2de None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:05.653844 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a76387-289c-4b30-9f6f-457e2c559d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:05.654532 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a76387-289c-4b30-9f6f-457e2c559d2e None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:05.654818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a76387-289c-4b30-9f6f-457e2c559d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:05.655101 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a76387-289c-4b30-9f6f-457e2c559d2e None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:05.656335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a76387-289c-4b30-9f6f-457e2c559d2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:05.657784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a76387-289c-4b30-9f6f-457e2c559d2e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:05.706730 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb2e32-5e65-4d00-ace4-87f8afc4e07f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:05.739215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31fdf6-b445-463e-b1ec-a3cc3d2cd310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:05.739841 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31fdf6-b445-463e-b1ec-a3cc3d2cd310 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:05.740130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31fdf6-b445-463e-b1ec-a3cc3d2cd310 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:05.740413 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31fdf6-b445-463e-b1ec-a3cc3d2cd310 None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:05.741540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31fdf6-b445-463e-b1ec-a3cc3d2cd310 None admin] RBAC: Authorizing `identity:get_project(project_id=19b75bd757df4c8cac09dd67b1f0ca8d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:05.743014 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31fdf6-b445-463e-b1ec-a3cc3d2cd310 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:05.744635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 39/84] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:05 2022] GET /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d => generated 298 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:05.745439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31fdf6-b445-463e-b1ec-a3cc3d2cd310 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:05.765896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 46/85] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 14 11:45:05 2022] POST /identity/v3/users => generated 289 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:45:05.765896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a76387-289c-4b30-9f6f-457e2c559d2e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:05.798113 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:05.861049 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31fdf6-b445-463e-b1ec-a3cc3d2cd310 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:05.863786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:05.873694 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a76387-289c-4b30-9f6f-457e2c559d2e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:05.901211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca1d73-aa9c-4f49-bc1a-8752255d8e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:05.901890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca1d73-aa9c-4f49-bc1a-8752255d8e32 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:05.902190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca1d73-aa9c-4f49-bc1a-8752255d8e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:05.902693 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca1d73-aa9c-4f49-bc1a-8752255d8e32 None admin] PATH_INFO: `/v3/users/d56b1b366e684a00aa34e438baa482b8` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:05.903660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca1d73-aa9c-4f49-bc1a-8752255d8e32 None admin] RBAC: Authorizing `identity:get_user(user_id=d56b1b366e684a00aa34e438baa482b8)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:05.905316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca1d73-aa9c-4f49-bc1a-8752255d8e32 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:05.912375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 40/86] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:05 2022] GET /identity/v3/users/d56b1b366e684a00aa34e438baa482b8 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:05.912375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca1d73-aa9c-4f49-bc1a-8752255d8e32 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:05.915520 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fe18a-3532-4e8a-a8c3-1042e470432c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:05.916257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fe18a-3532-4e8a-a8c3-1042e470432c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:05.916554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fe18a-3532-4e8a-a8c3-1042e470432c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:05.916926 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fe18a-3532-4e8a-a8c3-1042e470432c None admin] PATH_INFO: `/v3/users/d56b1b366e684a00aa34e438baa482b8` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:05.918174 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4fe18a-3532-4e8a-a8c3-1042e470432c None admin] RBAC: Authorizing `identity:get_user(user_id=d56b1b366e684a00aa34e438baa482b8)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:05.920174 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4fe18a-3532-4e8a-a8c3-1042e470432c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:05.922743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 47/87] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:05 2022] GET /identity/v3/users/d56b1b366e684a00aa34e438baa482b8 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:05.924146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fe18a-3532-4e8a-a8c3-1042e470432c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:06.031566 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca1d73-aa9c-4f49-bc1a-8752255d8e32 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:06.041969 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fe18a-3532-4e8a-a8c3-1042e470432c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:06.047877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:06.075392 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e424e3f-3fa2-4a85-9c81-c7854f2021cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:06.076145 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e424e3f-3fa2-4a85-9c81-c7854f2021cd None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:06.076468 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e424e3f-3fa2-4a85-9c81-c7854f2021cd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:06.076756 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e424e3f-3fa2-4a85-9c81-c7854f2021cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:06.083327 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a140607-787c-43a1-a1bd-61786286e47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:06.084201 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a140607-787c-43a1-a1bd-61786286e47c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:06.084667 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a140607-787c-43a1-a1bd-61786286e47c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:06.085131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a140607-787c-43a1-a1bd-61786286e47c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:06.086085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a140607-787c-43a1-a1bd-61786286e47c None admin] RBAC: Adding query filter params (role.id=d96ee0e293244ae6a04e8ce652aea68e, scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d, user.id=d56b1b366e684a00aa34e438baa482b8) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:06.086616 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a140607-787c-43a1-a1bd-61786286e47c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:06.088093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a140607-787c-43a1-a1bd-61786286e47c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:06.093798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 48/88] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:45:05 2022] GET /identity/v3/role_assignments?user.id=d56b1b366e684a00aa34e438baa482b8&scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d&role.id=d96ee0e293244ae6a04e8ce652aea68e => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:06.094661 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a140607-787c-43a1-a1bd-61786286e47c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:06.138429 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:06.178877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e424e3f-3fa2-4a85-9c81-c7854f2021cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:06.180394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e424e3f-3fa2-4a85-9c81-c7854f2021cd None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:06.228588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a140607-787c-43a1-a1bd-61786286e47c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:06.268703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca3ecd-d2c7-4d11-9994-0239904287ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:06.269331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca3ecd-d2c7-4d11-9994-0239904287ba None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:06.269630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca3ecd-d2c7-4d11-9994-0239904287ba None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:06.269957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca3ecd-d2c7-4d11-9994-0239904287ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:06.313568 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 41/89] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:06.314879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e424e3f-3fa2-4a85-9c81-c7854f2021cd None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:06.332616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:06.346149 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca3ecd-d2c7-4d11-9994-0239904287ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:06.347681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca3ecd-d2c7-4d11-9994-0239904287ba None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:06.373446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:06.428070 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e424e3f-3fa2-4a85-9c81-c7854f2021cd None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:06.466001 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6376761-377e-420f-add2-9f118345567a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:06.466629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6376761-377e-420f-add2-9f118345567a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:06.466917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6376761-377e-420f-add2-9f118345567a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:06.467547 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6376761-377e-420f-add2-9f118345567a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:06.514473 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 49/90] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:06.515311 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca3ecd-d2c7-4d11-9994-0239904287ba None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:06.544938 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6376761-377e-420f-add2-9f118345567a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:06.546995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6376761-377e-420f-add2-9f118345567a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:06.565986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:06.652725 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 42/91] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:06.653596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6376761-377e-420f-add2-9f118345567a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:06.757319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6376761-377e-420f-add2-9f118345567a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:06.774233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:06.779823 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca3ecd-d2c7-4d11-9994-0239904287ba None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:06.792399 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd7fc6-932b-4483-83c6-478d7a55f726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:06.793041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd7fc6-932b-4483-83c6-478d7a55f726 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:06.793324 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd7fc6-932b-4483-83c6-478d7a55f726 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:06.793605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd7fc6-932b-4483-83c6-478d7a55f726 None admin] PATH_INFO: `/v3/projects/b6871445408344a4b618cafb428583ce` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:06.794937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd7fc6-932b-4483-83c6-478d7a55f726 None admin] RBAC: Authorizing `identity:get_project(project_id=b6871445408344a4b618cafb428583ce)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:06.797581 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd7fc6-932b-4483-83c6-478d7a55f726 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:06.799366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 43/92] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:06 2022] GET /identity/v3/projects/b6871445408344a4b618cafb428583ce => generated 312 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:06.800036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd7fc6-932b-4483-83c6-478d7a55f726 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:06.811113 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:06.850157 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668834b-b11f-4fab-af00-6e2dc6890096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:06.850888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668834b-b11f-4fab-af00-6e2dc6890096 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:06.851183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668834b-b11f-4fab-af00-6e2dc6890096 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:06.851473 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668834b-b11f-4fab-af00-6e2dc6890096 None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:06.853956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668834b-b11f-4fab-af00-6e2dc6890096 None admin] RBAC: Authorizing `identity:get_project(project_id=19b75bd757df4c8cac09dd67b1f0ca8d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:06.855576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668834b-b11f-4fab-af00-6e2dc6890096 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:06.857418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 50/93] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:06 2022] GET /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d => generated 298 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:06.858313 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668834b-b11f-4fab-af00-6e2dc6890096 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:06.940331 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd7fc6-932b-4483-83c6-478d7a55f726 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:06.974425 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc436bb-52b3-485a-b46d-3bef388da443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:06.975065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc436bb-52b3-485a-b46d-3bef388da443 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:06.975350 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc436bb-52b3-485a-b46d-3bef388da443 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:06.975633 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc436bb-52b3-485a-b46d-3bef388da443 None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:06.976774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc436bb-52b3-485a-b46d-3bef388da443 None admin] RBAC: Authorizing `identity:get_project(project_id=19b75bd757df4c8cac09dd67b1f0ca8d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:06.978241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc436bb-52b3-485a-b46d-3bef388da443 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:06.979871 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 44/94] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:06 2022] GET /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:06.980656 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc436bb-52b3-485a-b46d-3bef388da443 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:07.042353 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:07.066969 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668834b-b11f-4fab-af00-6e2dc6890096 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:07.092268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc436bb-52b3-485a-b46d-3bef388da443 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:07.128281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bb158-1efc-4f7a-bcf3-5e8d1053e040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:07.128901 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bb158-1efc-4f7a-bcf3-5e8d1053e040 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:07.129182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bb158-1efc-4f7a-bcf3-5e8d1053e040 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:07.129458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bb158-1efc-4f7a-bcf3-5e8d1053e040 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:07.130220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bb158-1efc-4f7a-bcf3-5e8d1053e040 None admin] RBAC: Adding query filter params (role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d, user.id=6c2df9ec255e48ec9c7b4291f711f0cd) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:07.130540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bb158-1efc-4f7a-bcf3-5e8d1053e040 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:07.132065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bb158-1efc-4f7a-bcf3-5e8d1053e040 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:07.138436 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0862f-fef6-4a8a-93dc-2b52b5b2630c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:07.139330 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0862f-fef6-4a8a-93dc-2b52b5b2630c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:07.139822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0862f-fef6-4a8a-93dc-2b52b5b2630c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:07.140439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 45/95] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:45:06 2022] GET /identity/v3/role_assignments?user.id=6c2df9ec255e48ec9c7b4291f711f0cd&scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:07.141203 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0862f-fef6-4a8a-93dc-2b52b5b2630c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:07.141307 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bb158-1efc-4f7a-bcf3-5e8d1053e040 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:07.142383 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0862f-fef6-4a8a-93dc-2b52b5b2630c None admin] RBAC: Adding query filter params (role.id=d96ee0e293244ae6a04e8ce652aea68e, scope.project.id=b6871445408344a4b618cafb428583ce, user.id=d56b1b366e684a00aa34e438baa482b8) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:07.142907 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0862f-fef6-4a8a-93dc-2b52b5b2630c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:07.144833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0862f-fef6-4a8a-93dc-2b52b5b2630c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:07.159374 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 51/96] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:45:06 2022] GET /identity/v3/role_assignments?user.id=d56b1b366e684a00aa34e438baa482b8&scope.project.id=b6871445408344a4b618cafb428583ce&role.id=d96ee0e293244ae6a04e8ce652aea68e => generated 259 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:07.301272 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bb158-1efc-4f7a-bcf3-5e8d1053e040 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:07.322262 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:07.351565 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb58644-6a66-4edb-9a36-c30c4d27ae71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:07.352131 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb58644-6a66-4edb-9a36-c30c4d27ae71 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:07.352419 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb58644-6a66-4edb-9a36-c30c4d27ae71 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:07.352700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb58644-6a66-4edb-9a36-c30c4d27ae71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:07.353458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb58644-6a66-4edb-9a36-c30c4d27ae71 None admin] RBAC: Adding query filter params (role.id=1ab8c505dc344769abc347f66e529527, scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d, user.id=d56b1b366e684a00aa34e438baa482b8) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:07.353777 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb58644-6a66-4edb-9a36-c30c4d27ae71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:07.355312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb58644-6a66-4edb-9a36-c30c4d27ae71 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:07.366983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 46/97] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:45:07 2022] GET /identity/v3/role_assignments?user.id=d56b1b366e684a00aa34e438baa482b8&scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d&role.id=1ab8c505dc344769abc347f66e529527 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:08.608306 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2f75b-7d49-43cd-bbce-12331953985a None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:08.609393 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2f75b-7d49-43cd-bbce-12331953985a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:08.609912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2f75b-7d49-43cd-bbce-12331953985a None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:08.611726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 52/98] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:08.616856 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbd429-84c5-441e-9277-da64387e2612 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:08.617815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbd429-84c5-441e-9277-da64387e2612 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:08.618373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbd429-84c5-441e-9277-da64387e2612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:08.646887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-75bbd429-84c5-441e-9277-da64387e2612 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:08.648451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-75bbd429-84c5-441e-9277-da64387e2612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:08.724859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 47/99] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:08.930260 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b9624-6bbe-4afc-8a47-8feb73adc86c None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:08.930893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b9624-6bbe-4afc-8a47-8feb73adc86c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:08.931420 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b9624-6bbe-4afc-8a47-8feb73adc86c None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:08.933344 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 53/100] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:08.940805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8e962-9cac-466d-b907-1fedf65aa930 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:08.941481 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8e962-9cac-466d-b907-1fedf65aa930 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:08.941991 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8e962-9cac-466d-b907-1fedf65aa930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:08.968860 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-adb8e962-9cac-466d-b907-1fedf65aa930 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:08.970109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-adb8e962-9cac-466d-b907-1fedf65aa930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:09.079170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 48/101] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:45:09.152349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67dd03-b97c-415b-b03a-2c422a09d48b None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:09.152679 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67dd03-b97c-415b-b03a-2c422a09d48b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:09.152935 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67dd03-b97c-415b-b03a-2c422a09d48b None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:09.154324 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 54/102] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:09.160167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7c346-6859-4690-bd1d-cc2e507381fa None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:09.160742 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7c346-6859-4690-bd1d-cc2e507381fa None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:09.161220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7c346-6859-4690-bd1d-cc2e507381fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:09.170303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67dd03-b97c-415b-b03a-2c422a09d48b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:09.186185 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-cea7c346-6859-4690-bd1d-cc2e507381fa None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:09.187684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cea7c346-6859-4690-bd1d-cc2e507381fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:09.223009 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:09.284279 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:09.297696 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 49/103] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:09.318541 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67dd03-b97c-415b-b03a-2c422a09d48b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:09.367184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603f151-9c5b-4284-96b7-6512a11c7186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:09.367840 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603f151-9c5b-4284-96b7-6512a11c7186 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:09.368224 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603f151-9c5b-4284-96b7-6512a11c7186 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:09.368515 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603f151-9c5b-4284-96b7-6512a11c7186 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:09.379269 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0603f151-9c5b-4284-96b7-6512a11c7186 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:09.384729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-0603f151-9c5b-4284-96b7-6512a11c7186 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 11:45:09.384729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:09.384729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:09.384729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:09.384729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:09.384729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:09.384729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:09.384729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:09.384729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:09.384729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:09.384729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:09.384729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:09.384729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:09.384729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:09.384729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:09.384729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:09.385663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:09.385663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:09.385663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:09.385663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:09.385663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:09.385663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:09.385663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:09.385663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:09.385663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:09.385663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:09.385663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:09.385663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:09.385663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:09.385663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:09.385663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:09.385663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:09.386832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:09.386832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:09.386832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:09.386832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:09.386832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:09.386832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:09.386832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:09.386832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:09.386832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:09.386832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:09.386832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 11:45:09.386832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:09.386832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 55/104] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:45:09.394050 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7c346-6859-4690-bd1d-cc2e507381fa None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:09.518453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7c346-6859-4690-bd1d-cc2e507381fa None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:09.569641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adcad5d-4eca-4908-9443-a7b4816ba513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:09.570305 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcad5d-4eca-4908-9443-a7b4816ba513 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:09.570593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcad5d-4eca-4908-9443-a7b4816ba513 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:09.570876 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcad5d-4eca-4908-9443-a7b4816ba513 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:09.571550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adcad5d-4eca-4908-9443-a7b4816ba513 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:09.571858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adcad5d-4eca-4908-9443-a7b4816ba513 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:09.573248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adcad5d-4eca-4908-9443-a7b4816ba513 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:09.585446 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 50/105] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:09.591200 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603f151-9c5b-4284-96b7-6512a11c7186 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:09.668199 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adcad5d-4eca-4908-9443-a7b4816ba513 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:09.718765 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:09.760541 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603f151-9c5b-4284-96b7-6512a11c7186 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:09.786256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:09.830652 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97e4e8-5641-438a-82e2-287bf7be3b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:09.831315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97e4e8-5641-438a-82e2-287bf7be3b09 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:09.831605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97e4e8-5641-438a-82e2-287bf7be3b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:09.831887 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97e4e8-5641-438a-82e2-287bf7be3b09 None admin] PATH_INFO: `/v3/users/a737e413d8744fc49876566629ebfe18` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:09.854291 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adcad5d-4eca-4908-9443-a7b4816ba513 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:09.860635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97e4e8-5641-438a-82e2-287bf7be3b09 None admin] RBAC: Authorizing `identity:get_user(user_id=a737e413d8744fc49876566629ebfe18)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:09.862168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97e4e8-5641-438a-82e2-287bf7be3b09 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:09.866700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 56/106] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:09 2022] GET /identity/v3/users/a737e413d8744fc49876566629ebfe18 => generated 289 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:09.867508 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97e4e8-5641-438a-82e2-287bf7be3b09 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:09.905494 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0b608-90e5-414a-9327-05e92519d49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:09.906187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0b608-90e5-414a-9327-05e92519d49a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:09.906641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0b608-90e5-414a-9327-05e92519d49a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:09.907069 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0b608-90e5-414a-9327-05e92519d49a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:09.908937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0b608-90e5-414a-9327-05e92519d49a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:09.910633 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0b608-90e5-414a-9327-05e92519d49a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:09.912588 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 51/107] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:45:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:09.913749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0b608-90e5-414a-9327-05e92519d49a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:10.072728 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97e4e8-5641-438a-82e2-287bf7be3b09 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:10.079398 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:10.085473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0b608-90e5-414a-9327-05e92519d49a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:10.110353 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c2b58-a348-45e9-91f6-c8621e02360a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:10.111240 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c2b58-a348-45e9-91f6-c8621e02360a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:10.111979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c2b58-a348-45e9-91f6-c8621e02360a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:10.112407 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c2b58-a348-45e9-91f6-c8621e02360a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:10.125414 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c2b58-a348-45e9-91f6-c8621e02360a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:10.130911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-234c2b58-a348-45e9-91f6-c8621e02360a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:10.130911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:10.130911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:10.130911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:10.130911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:10.130911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:10.130911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:10.130911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:10.130911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:10.130911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:10.130911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:10.130911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:10.130911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:10.130911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:10.130911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:10.130911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:10.131755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:10.131755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:10.131755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:10.131755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:10.131755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:10.131755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:10.131755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:10.131755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:10.131755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:10.131755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:10.131755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:10.131755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:10.131755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:10.131755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:10.131755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:10.131755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:10.132514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:10.132514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:10.132514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:10.132514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:10.132514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:10.132514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:10.132514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:10.132514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:10.132514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:10.132514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:10.132514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:10.132514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:10.135740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 57/108] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:10.136857 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c2b58-a348-45e9-91f6-c8621e02360a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:10.161732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76636098-7146-46f8-b261-a21a66666c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:10.162644 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76636098-7146-46f8-b261-a21a66666c61 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:10.163107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76636098-7146-46f8-b261-a21a66666c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:10.163551 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76636098-7146-46f8-b261-a21a66666c61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:10.178313 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:10.237048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76636098-7146-46f8-b261-a21a66666c61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:10.239052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76636098-7146-46f8-b261-a21a66666c61 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:10.255988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c2b58-a348-45e9-91f6-c8621e02360a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:10.294043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07d0cc-68a8-48d3-a55c-390f4a94863b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:10.294765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07d0cc-68a8-48d3-a55c-390f4a94863b None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:10.295216 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07d0cc-68a8-48d3-a55c-390f4a94863b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:10.295670 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07d0cc-68a8-48d3-a55c-390f4a94863b None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:10.297103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07d0cc-68a8-48d3-a55c-390f4a94863b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:10.298859 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07d0cc-68a8-48d3-a55c-390f4a94863b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:10.348774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:10.360475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 52/109] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:10.360475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76636098-7146-46f8-b261-a21a66666c61 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:10.388354 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 58/110] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 14 11:45:10 2022] POST /identity/v3/projects => generated 302 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:45:10.399558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e56a5-db58-4001-84dc-d286f61ab980 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:10.400158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e56a5-db58-4001-84dc-d286f61ab980 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:10.400601 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e56a5-db58-4001-84dc-d286f61ab980 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:10.402243 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 59/111] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:10.403156 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e56a5-db58-4001-84dc-d286f61ab980 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:10.490428 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76636098-7146-46f8-b261-a21a66666c61 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:10.541171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c84ae4-9bf0-435e-ad51-372a65dff1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:10.542078 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c84ae4-9bf0-435e-ad51-372a65dff1ee None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:10.542589 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c84ae4-9bf0-435e-ad51-372a65dff1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:10.543528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c84ae4-9bf0-435e-ad51-372a65dff1ee None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:10.544562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c84ae4-9bf0-435e-ad51-372a65dff1ee None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:10.545017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c84ae4-9bf0-435e-ad51-372a65dff1ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:10.547190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c84ae4-9bf0-435e-ad51-372a65dff1ee None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:10.558228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 53/112] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:10.560687 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ff358-50b6-48d4-9289-8719682cefe1 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:10.561383 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ff358-50b6-48d4-9289-8719682cefe1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:10.561921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ff358-50b6-48d4-9289-8719682cefe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:10.584627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-455ff358-50b6-48d4-9289-8719682cefe1 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:10.586431 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-455ff358-50b6-48d4-9289-8719682cefe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:10.631598 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:10.654220 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e56a5-db58-4001-84dc-d286f61ab980 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:10.656814 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:10.676448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 54/113] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:10.677038 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ff358-50b6-48d4-9289-8719682cefe1 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:10.696961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26156161-6dd2-45d2-b7ed-1efcefe8acf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:10.697607 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26156161-6dd2-45d2-b7ed-1efcefe8acf6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:10.698669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26156161-6dd2-45d2-b7ed-1efcefe8acf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:10.698669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26156161-6dd2-45d2-b7ed-1efcefe8acf6 None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:10.700163 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26156161-6dd2-45d2-b7ed-1efcefe8acf6 None admin] RBAC: Authorizing `identity:get_project(project_id=19b75bd757df4c8cac09dd67b1f0ca8d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:10.701362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26156161-6dd2-45d2-b7ed-1efcefe8acf6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:10.702906 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 60/114] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:10 2022] GET /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:10.704453 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17392498-19d1-4365-94be-468ecc0b826e None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:10.704453 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17392498-19d1-4365-94be-468ecc0b826e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:10.704610 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17392498-19d1-4365-94be-468ecc0b826e None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:10.706015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 61/115] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:10.706936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f23ec-7112-45cd-90fd-5e4fd298a16a None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:10.707208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f23ec-7112-45cd-90fd-5e4fd298a16a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:10.707485 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f23ec-7112-45cd-90fd-5e4fd298a16a None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:10.708685 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 62/116] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:10.713685 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f23ec-7112-45cd-90fd-5e4fd298a16a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:10.803766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ff358-50b6-48d4-9289-8719682cefe1 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:10.841043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f23ec-7112-45cd-90fd-5e4fd298a16a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:10.846477 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2366848-5802-4955-b013-da8ab6780207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:10.847130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2366848-5802-4955-b013-da8ab6780207 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:10.847476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2366848-5802-4955-b013-da8ab6780207 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:10.847764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2366848-5802-4955-b013-da8ab6780207 None admin] PATH_INFO: `/v3/users/d56b1b366e684a00aa34e438baa482b8` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:10.849028 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2366848-5802-4955-b013-da8ab6780207 None admin] RBAC: Authorizing `identity:get_user(user_id=d56b1b366e684a00aa34e438baa482b8)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:10.850988 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2366848-5802-4955-b013-da8ab6780207 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:10.853000 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 55/117] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:10 2022] GET /identity/v3/users/d56b1b366e684a00aa34e438baa482b8 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:10.854930 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c6a9b-9764-4e47-88c0-732d771086b6 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:10.855220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c6a9b-9764-4e47-88c0-732d771086b6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:10.855503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c6a9b-9764-4e47-88c0-732d771086b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:10.875262 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f21c6a9b-9764-4e47-88c0-732d771086b6 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:10.876462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f21c6a9b-9764-4e47-88c0-732d771086b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:10.878724 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bc266-ed7b-4f10-9bcf-d3c3e1e8ea9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:10.879400 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bc266-ed7b-4f10-9bcf-d3c3e1e8ea9a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:10.879696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bc266-ed7b-4f10-9bcf-d3c3e1e8ea9a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:10.879981 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bc266-ed7b-4f10-9bcf-d3c3e1e8ea9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:10.880727 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bc266-ed7b-4f10-9bcf-d3c3e1e8ea9a None admin] RBAC: Adding query filter params (role.id=f792865844594321a8a67b38335cdccb, scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d, user.id=a737e413d8744fc49876566629ebfe18) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:10.881043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bc266-ed7b-4f10-9bcf-d3c3e1e8ea9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:10.882587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bc266-ed7b-4f10-9bcf-d3c3e1e8ea9a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:10.888434 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 63/118] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:45:10 2022] GET /identity/v3/role_assignments?user.id=a737e413d8744fc49876566629ebfe18&scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d&role.id=f792865844594321a8a67b38335cdccb => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:10.889453 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ea4ab-85a5-468f-81a2-8ce6a94f36ac None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:10.889753 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ea4ab-85a5-468f-81a2-8ce6a94f36ac None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:10.890585 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ea4ab-85a5-468f-81a2-8ce6a94f36ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:10.909921 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b66ea4ab-85a5-468f-81a2-8ce6a94f36ac None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:10.911459 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b66ea4ab-85a5-468f-81a2-8ce6a94f36ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:10.923785 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:10.978636 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:10.990304 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 64/119] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:10.991192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ea4ab-85a5-468f-81a2-8ce6a94f36ac None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:10.999820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 56/120] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:11.000712 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c6a9b-9764-4e47-88c0-732d771086b6 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:11.165040 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c6a9b-9764-4e47-88c0-732d771086b6 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:11.190731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ea4ab-85a5-468f-81a2-8ce6a94f36ac None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:11.232105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a5ccd-7fa2-4e29-8d32-df9fbf867c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:11.232763 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a5ccd-7fa2-4e29-8d32-df9fbf867c4f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:11.233060 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a5ccd-7fa2-4e29-8d32-df9fbf867c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:11.233350 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a5ccd-7fa2-4e29-8d32-df9fbf867c4f None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:11.235102 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a5ccd-7fa2-4e29-8d32-df9fbf867c4f None admin] RBAC: Authorizing `identity:get_project(project_id=19b75bd757df4c8cac09dd67b1f0ca8d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:11.236588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a5ccd-7fa2-4e29-8d32-df9fbf867c4f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:11.238463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 65/121] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:10 2022] GET /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:11.245897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a5ccd-7fa2-4e29-8d32-df9fbf867c4f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:11.255076 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b33057-447a-47ab-b54e-3a529699a23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:11.255852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b33057-447a-47ab-b54e-3a529699a23c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:11.256145 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b33057-447a-47ab-b54e-3a529699a23c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:11.256430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b33057-447a-47ab-b54e-3a529699a23c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:11.274227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:11.274637 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b33057-447a-47ab-b54e-3a529699a23c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:11.284811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-49b33057-447a-47ab-b54e-3a529699a23c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:45:11.284811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:11.284811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:11.284811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:11.284811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:11.284811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:11.284811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:11.284811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:11.284811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:11.284811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:11.284811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:11.284811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:11.284811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:11.284811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:11.284811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:11.284811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:11.285676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:11.285676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:11.285676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:11.285676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:11.285676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:11.285676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:11.285676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:11.285676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:11.285676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:11.285676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:11.285676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:11.285676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:11.285676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:11.285676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:11.285676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:11.285676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:11.286571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:11.286571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:11.286571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:11.286571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:11.286571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:11.286571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:11.286571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:11.286571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:11.286571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:11.286571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:11.286571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:45:11.286571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:11.289569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 57/122] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:45:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:11.295482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b33057-447a-47ab-b54e-3a529699a23c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:11.368864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:11.381317 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a5ccd-7fa2-4e29-8d32-df9fbf867c4f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:11.418552 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2532b4-710b-4740-bc2f-5b0c0741c65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:11.419779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2532b4-710b-4740-bc2f-5b0c0741c65f None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:11.420269 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2532b4-710b-4740-bc2f-5b0c0741c65f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:11.420747 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2532b4-710b-4740-bc2f-5b0c0741c65f None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d/users/d56b1b366e684a00aa34e438baa482b8/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:11.424460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2532b4-710b-4740-bc2f-5b0c0741c65f None admin] RBAC: Authorizing `identity:create_grant(project_id=19b75bd757df4c8cac09dd67b1f0ca8d, user_id=d56b1b366e684a00aa34e438baa482b8, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:11.426485 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2532b4-710b-4740-bc2f-5b0c0741c65f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:11.464924 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b33057-447a-47ab-b54e-3a529699a23c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:11.490135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 66/123] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:45:11 2022] PUT /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d/users/d56b1b366e684a00aa34e438baa482b8/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 11:45:11.491315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2532b4-710b-4740-bc2f-5b0c0741c65f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:11.526916 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06384fc6-e52d-4d2d-a9c6-8b640ff254e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:11.529256 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06384fc6-e52d-4d2d-a9c6-8b640ff254e1 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:11.530187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06384fc6-e52d-4d2d-a9c6-8b640ff254e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:11.532312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06384fc6-e52d-4d2d-a9c6-8b640ff254e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:11.533294 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06384fc6-e52d-4d2d-a9c6-8b640ff254e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:11.533814 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06384fc6-e52d-4d2d-a9c6-8b640ff254e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:11.536973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06384fc6-e52d-4d2d-a9c6-8b640ff254e1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:11.557372 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 58/124] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:11.565054 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06384fc6-e52d-4d2d-a9c6-8b640ff254e1 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:11.670394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:11.717233 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2532b4-710b-4740-bc2f-5b0c0741c65f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:11.745581 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06384fc6-e52d-4d2d-a9c6-8b640ff254e1 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:11.758229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:11.760773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d23be1-6ade-4e55-a685-4afda6727401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:11.761687 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d23be1-6ade-4e55-a685-4afda6727401 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:11.762013 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d23be1-6ade-4e55-a685-4afda6727401 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:11.762297 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d23be1-6ade-4e55-a685-4afda6727401 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:11.775064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d23be1-6ade-4e55-a685-4afda6727401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:11.783131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-89d23be1-6ade-4e55-a685-4afda6727401 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:11.783131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:11.783131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:11.783131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:11.783131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:11.783131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:11.783131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:11.783131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:11.783131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:11.783131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:11.783131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:11.783131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:11.783131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:11.783131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:11.783131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:11.783131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:11.784057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:11.784057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:11.784057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:11.784057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:11.784057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:11.784057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:11.784057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:11.784057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:11.784057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:11.784057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:11.784057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:11.784057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:11.784057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:11.784057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:11.784057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:11.784057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:11.784841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:11.784841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:11.784841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:11.784841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:11.784841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:11.784841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:11.784841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:11.784841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:11.784841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:11.784841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:11.784841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:11.784841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:11.784841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 67/125] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:11.785479 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d23be1-6ade-4e55-a685-4afda6727401 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:11.796704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c981e-de34-4cc0-96d1-af3ae2860185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:11.797668 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c981e-de34-4cc0-96d1-af3ae2860185 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:11.798198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c981e-de34-4cc0-96d1-af3ae2860185 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:11.798681 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c981e-de34-4cc0-96d1-af3ae2860185 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:11.811368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c981e-de34-4cc0-96d1-af3ae2860185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:11.816921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-934c981e-de34-4cc0-96d1-af3ae2860185 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:11.816921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:11.816921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:11.816921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:11.816921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:11.816921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:11.816921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:11.816921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:11.816921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:11.816921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:11.816921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:11.816921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:11.816921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:11.816921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:11.816921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:11.816921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:11.817792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:11.817792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:11.817792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:11.817792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:11.817792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:11.817792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:11.817792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:11.817792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:11.817792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:11.817792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:11.817792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:11.817792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:11.817792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:11.817792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:11.817792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:11.817792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:11.818597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:11.818597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:11.818597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:11.818597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:11.818597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:11.818597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:11.818597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:11.818597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:11.818597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:11.818597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:11.818597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:11.818597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:11.829592 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 59/126] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:45:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:11.830710 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c981e-de34-4cc0-96d1-af3ae2860185 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:11.947734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c981e-de34-4cc0-96d1-af3ae2860185 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:11.979915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d23be1-6ade-4e55-a685-4afda6727401 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:11.995227 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23045a59-e605-4217-9bde-2a2b0f1d4251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:11.996325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23045a59-e605-4217-9bde-2a2b0f1d4251 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:11.996810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23045a59-e605-4217-9bde-2a2b0f1d4251 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:11.997286 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23045a59-e605-4217-9bde-2a2b0f1d4251 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:11.998201 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23045a59-e605-4217-9bde-2a2b0f1d4251 None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:11.998844 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23045a59-e605-4217-9bde-2a2b0f1d4251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:12.001335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23045a59-e605-4217-9bde-2a2b0f1d4251 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:12.011104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 60/127] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:12.012213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23045a59-e605-4217-9bde-2a2b0f1d4251 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:12.023842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe5493-d70d-4b4d-a452-d983bd6f6579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:12.024492 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe5493-d70d-4b4d-a452-d983bd6f6579 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:12.024803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe5493-d70d-4b4d-a452-d983bd6f6579 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:12.025088 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe5493-d70d-4b4d-a452-d983bd6f6579 None admin] PATH_INFO: `/v3/users/6c2df9ec255e48ec9c7b4291f711f0cd` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:12.026272 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe5493-d70d-4b4d-a452-d983bd6f6579 None admin] RBAC: Authorizing `identity:get_user(user_id=6c2df9ec255e48ec9c7b4291f711f0cd)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:12.027757 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe5493-d70d-4b4d-a452-d983bd6f6579 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:12.029357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 68/128] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:11 2022] GET /identity/v3/users/6c2df9ec255e48ec9c7b4291f711f0cd => generated 247 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:12.033023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe5493-d70d-4b4d-a452-d983bd6f6579 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:12.033660 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:12.060548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:12.161008 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23045a59-e605-4217-9bde-2a2b0f1d4251 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:12.190815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe5493-d70d-4b4d-a452-d983bd6f6579 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:12.230700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b21e00-7586-47c6-912b-33da94b5f159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:12.231655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b21e00-7586-47c6-912b-33da94b5f159 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:12.232169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b21e00-7586-47c6-912b-33da94b5f159 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:12.232647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b21e00-7586-47c6-912b-33da94b5f159 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:12.233513 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b21e00-7586-47c6-912b-33da94b5f159 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:12.234321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b21e00-7586-47c6-912b-33da94b5f159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:12.236941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b21e00-7586-47c6-912b-33da94b5f159 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:12.250151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 61/129] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:45:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:12.251060 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b21e00-7586-47c6-912b-33da94b5f159 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:12.257478 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a1947-ccf4-4eac-a34a-b0771fdc0da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:12.258411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a1947-ccf4-4eac-a34a-b0771fdc0da8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:12.259010 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a1947-ccf4-4eac-a34a-b0771fdc0da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:12.259496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a1947-ccf4-4eac-a34a-b0771fdc0da8 None admin] PATH_INFO: `/v3/users/d56b1b366e684a00aa34e438baa482b8` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:12.260903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a1947-ccf4-4eac-a34a-b0771fdc0da8 None admin] RBAC: Authorizing `identity:get_user(user_id=d56b1b366e684a00aa34e438baa482b8)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:12.262581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a1947-ccf4-4eac-a34a-b0771fdc0da8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:12.264394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 69/130] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:12 2022] GET /identity/v3/users/d56b1b366e684a00aa34e438baa482b8 => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:12.266165 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a1947-ccf4-4eac-a34a-b0771fdc0da8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:12.390211 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:12.414719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:12.421906 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b21e00-7586-47c6-912b-33da94b5f159 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:12.451419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a1947-ccf4-4eac-a34a-b0771fdc0da8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:12.461007 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1274e-e2e3-466e-a24e-67aec7ee575f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:12.461661 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1274e-e2e3-466e-a24e-67aec7ee575f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:12.461979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1274e-e2e3-466e-a24e-67aec7ee575f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:12.462262 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1274e-e2e3-466e-a24e-67aec7ee575f None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:12.463822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1274e-e2e3-466e-a24e-67aec7ee575f None admin] RBAC: Authorizing `identity:get_project(project_id=19b75bd757df4c8cac09dd67b1f0ca8d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:12.465295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1274e-e2e3-466e-a24e-67aec7ee575f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:12.466964 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 62/131] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:12 2022] GET /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:12.468351 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1274e-e2e3-466e-a24e-67aec7ee575f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:12.535616 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d7a57-09bc-4012-9164-b90ea33e5998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:12.536194 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d7a57-09bc-4012-9164-b90ea33e5998 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:12.536486 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d7a57-09bc-4012-9164-b90ea33e5998 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:12.536768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d7a57-09bc-4012-9164-b90ea33e5998 None admin] PATH_INFO: `/v3/users/d56b1b366e684a00aa34e438baa482b8` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:12.537949 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d7a57-09bc-4012-9164-b90ea33e5998 None admin] RBAC: Authorizing `identity:get_user(user_id=d56b1b366e684a00aa34e438baa482b8)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:12.539420 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d7a57-09bc-4012-9164-b90ea33e5998 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:12.541024 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 70/132] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:12 2022] GET /identity/v3/users/d56b1b366e684a00aa34e438baa482b8 => generated 275 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:12.541778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d7a57-09bc-4012-9164-b90ea33e5998 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:12.589976 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1274e-e2e3-466e-a24e-67aec7ee575f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:12.628532 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43fead-0b3b-4d9b-8c5b-1f50e876f4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:12.629222 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43fead-0b3b-4d9b-8c5b-1f50e876f4fe None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:12.629519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43fead-0b3b-4d9b-8c5b-1f50e876f4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:12.629804 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43fead-0b3b-4d9b-8c5b-1f50e876f4fe None admin] PATH_INFO: `/v3/projects/b6871445408344a4b618cafb428583ce` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:12.633117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43fead-0b3b-4d9b-8c5b-1f50e876f4fe None admin] RBAC: Authorizing `identity:get_project(project_id=b6871445408344a4b618cafb428583ce)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:12.634659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43fead-0b3b-4d9b-8c5b-1f50e876f4fe None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:12.637339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 63/133] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:12 2022] GET /identity/v3/projects/b6871445408344a4b618cafb428583ce => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:12.638074 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43fead-0b3b-4d9b-8c5b-1f50e876f4fe None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:12.678345 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:12.748658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d7a57-09bc-4012-9164-b90ea33e5998 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:12.803368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871e148-640a-4185-90b5-1167c7ef6a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:12.803966 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871e148-640a-4185-90b5-1167c7ef6a0c None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:12.804254 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871e148-640a-4185-90b5-1167c7ef6a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:12.804541 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871e148-640a-4185-90b5-1167c7ef6a0c None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d/users/6c2df9ec255e48ec9c7b4291f711f0cd/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:12.807551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871e148-640a-4185-90b5-1167c7ef6a0c None admin] RBAC: Authorizing `identity:create_grant(project_id=19b75bd757df4c8cac09dd67b1f0ca8d, user_id=6c2df9ec255e48ec9c7b4291f711f0cd, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:12.809027 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871e148-640a-4185-90b5-1167c7ef6a0c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:12.825956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43fead-0b3b-4d9b-8c5b-1f50e876f4fe None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:12.865041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee2974-680a-43a2-a021-094e238b5c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:12.865731 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee2974-680a-43a2-a021-094e238b5c4b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:12.866051 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee2974-680a-43a2-a021-094e238b5c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:12.866335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee2974-680a-43a2-a021-094e238b5c4b None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:12.867464 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee2974-680a-43a2-a021-094e238b5c4b None admin] RBAC: Authorizing `identity:get_project(project_id=19b75bd757df4c8cac09dd67b1f0ca8d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:12.868977 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee2974-680a-43a2-a021-094e238b5c4b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:12.870826 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 64/134] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:12 2022] GET /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:12.871515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee2974-680a-43a2-a021-094e238b5c4b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:12.913602 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:12.918175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 71/135] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:45:12 2022] PUT /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d/users/6c2df9ec255e48ec9c7b4291f711f0cd/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:45:12.919048 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871e148-640a-4185-90b5-1167c7ef6a0c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:13.033426 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:13.047915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee2974-680a-43a2-a021-094e238b5c4b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:13.123628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871e148-640a-4185-90b5-1167c7ef6a0c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:13.127792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd983f99-6ee0-4628-88c0-07f277771897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:13.128485 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd983f99-6ee0-4628-88c0-07f277771897 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:13.128773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd983f99-6ee0-4628-88c0-07f277771897 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:13.129055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd983f99-6ee0-4628-88c0-07f277771897 None admin] PATH_INFO: `/v3/projects/b6871445408344a4b618cafb428583ce/users/d56b1b366e684a00aa34e438baa482b8/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:13.134574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd983f99-6ee0-4628-88c0-07f277771897 None admin] RBAC: Authorizing `identity:create_grant(project_id=b6871445408344a4b618cafb428583ce, user_id=d56b1b366e684a00aa34e438baa482b8, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:13.136183 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd983f99-6ee0-4628-88c0-07f277771897 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:13.162043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723038e5-e721-4bc8-91b5-a9d08e8c938f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:13.162733 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723038e5-e721-4bc8-91b5-a9d08e8c938f None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:13.163119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723038e5-e721-4bc8-91b5-a9d08e8c938f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:13.163408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723038e5-e721-4bc8-91b5-a9d08e8c938f None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d/users/d56b1b366e684a00aa34e438baa482b8/roles/1ab8c505dc344769abc347f66e529527` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:13.165887 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723038e5-e721-4bc8-91b5-a9d08e8c938f None admin] RBAC: Authorizing `identity:create_grant(project_id=19b75bd757df4c8cac09dd67b1f0ca8d, user_id=d56b1b366e684a00aa34e438baa482b8, role_id=1ab8c505dc344769abc347f66e529527)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:13.167398 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723038e5-e721-4bc8-91b5-a9d08e8c938f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:13.206374 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 65/136] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:45:12 2022] PUT /identity/v3/projects/b6871445408344a4b618cafb428583ce/users/d56b1b366e684a00aa34e438baa482b8/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:45:13.207756 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e529f-5b2d-4a16-9457-6b53dcc63b47 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:13.208055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e529f-5b2d-4a16-9457-6b53dcc63b47 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:13.208336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e529f-5b2d-4a16-9457-6b53dcc63b47 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:13.210186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 66/137] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:13.220067 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b4f77-80aa-4985-9d87-d916104712be None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:13.220450 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b4f77-80aa-4985-9d87-d916104712be None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:13.220734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b4f77-80aa-4985-9d87-d916104712be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:13.241217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8f6b4f77-80aa-4985-9d87-d916104712be None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:13.242513 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8f6b4f77-80aa-4985-9d87-d916104712be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:13.255483 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 72/138] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:45:12 2022] PUT /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d/users/d56b1b366e684a00aa34e438baa482b8/roles/1ab8c505dc344769abc347f66e529527 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:45:13.329808 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 67/139] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:13.651581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc34b26-9bf7-483d-8b09-c4776ceb8f24 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:13.651944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc34b26-9bf7-483d-8b09-c4776ceb8f24 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:13.652150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc34b26-9bf7-483d-8b09-c4776ceb8f24 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:13.653516 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 73/140] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:13.659207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca0c83-f6b0-4501-8c58-6f97af8c47d4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:13.659500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca0c83-f6b0-4501-8c58-6f97af8c47d4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:13.659774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca0c83-f6b0-4501-8c58-6f97af8c47d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:13.679455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-aaca0c83-f6b0-4501-8c58-6f97af8c47d4 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:13.680649 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-aaca0c83-f6b0-4501-8c58-6f97af8c47d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:13.760395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 68/141] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:13.783666 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc34b26-9bf7-483d-8b09-c4776ceb8f24 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:13.872382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:13.934599 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc34b26-9bf7-483d-8b09-c4776ceb8f24 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:13.985450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252330f-7ff2-4077-93db-0e4515284e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:13.986131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252330f-7ff2-4077-93db-0e4515284e74 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:13.986418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252330f-7ff2-4077-93db-0e4515284e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:13.986699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252330f-7ff2-4077-93db-0e4515284e74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:13.987842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b252330f-7ff2-4077-93db-0e4515284e74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:13.989314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b252330f-7ff2-4077-93db-0e4515284e74 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:13.991130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 74/142] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:45:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:13.996951 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca0c83-f6b0-4501-8c58-6f97af8c47d4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:14.025660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:14.067425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252330f-7ff2-4077-93db-0e4515284e74 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:14.174502 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca0c83-f6b0-4501-8c58-6f97af8c47d4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:14.236016 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce93b4-51bb-439e-a545-209bc84d3c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:14.236546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce93b4-51bb-439e-a545-209bc84d3c4a None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:14.236832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce93b4-51bb-439e-a545-209bc84d3c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:14.237116 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce93b4-51bb-439e-a545-209bc84d3c4a None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:14.237792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bce93b4-51bb-439e-a545-209bc84d3c4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:14.239258 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bce93b4-51bb-439e-a545-209bc84d3c4a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:14.279820 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252330f-7ff2-4077-93db-0e4515284e74 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:14.308507 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:14.318559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5812a7f9-b3f2-421e-962f-388944603c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:14.319476 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812a7f9-b3f2-421e-962f-388944603c90 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:14.319936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812a7f9-b3f2-421e-962f-388944603c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:14.320431 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812a7f9-b3f2-421e-962f-388944603c90 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:14.332312 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5812a7f9-b3f2-421e-962f-388944603c90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:14.337893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-5812a7f9-b3f2-421e-962f-388944603c90 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 11:45:14.337893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:14.337893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:14.337893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:14.337893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:14.337893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:14.337893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:14.337893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:14.337893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:14.337893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:14.337893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:14.337893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:14.337893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:14.337893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:14.337893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:14.337893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:14.338734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:14.338734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:14.338734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:14.338734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:14.338734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:14.338734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:14.338734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:14.338734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:14.338734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:14.338734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:14.338734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:14.338734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:14.338734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:14.338734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:14.338734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:14.338734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:14.339549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:14.339549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:14.339549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:14.339549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:14.339549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:14.339549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:14.339549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:14.339549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:14.339549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:14.339549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:14.339549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 11:45:14.339549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:14.341792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 75/143] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:45:14.343197 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43765bc0-1f46-487b-a082-085bbf1b23f4 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:14.343645 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43765bc0-1f46-487b-a082-085bbf1b23f4 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:14.344163 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43765bc0-1f46-487b-a082-085bbf1b23f4 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:14.345687 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 76/144] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:14.347802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43765bc0-1f46-487b-a082-085bbf1b23f4 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:14.370016 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 69/145] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 14 11:45:13 2022] POST /identity/v3/users => generated 283 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:45:14.371525 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265556be-4d40-4b8a-a843-cbdd55773dfa None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:14.371959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265556be-4d40-4b8a-a843-cbdd55773dfa None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:14.372881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265556be-4d40-4b8a-a843-cbdd55773dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:14.408865 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-265556be-4d40-4b8a-a843-cbdd55773dfa None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:14.412122 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-265556be-4d40-4b8a-a843-cbdd55773dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:14.578295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 70/146] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:14.582531 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43765bc0-1f46-487b-a082-085bbf1b23f4 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:14.629681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4ae0c-6ac5-4824-9b4f-041cbc22307e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:14.630553 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4ae0c-6ac5-4824-9b4f-041cbc22307e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:14.630972 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4ae0c-6ac5-4824-9b4f-041cbc22307e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:14.631388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4ae0c-6ac5-4824-9b4f-041cbc22307e None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:14.632288 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4ae0c-6ac5-4824-9b4f-041cbc22307e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:14.632737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4ae0c-6ac5-4824-9b4f-041cbc22307e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:14.634307 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4ae0c-6ac5-4824-9b4f-041cbc22307e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:14.646946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 77/147] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:14.652719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265556be-4d40-4b8a-a843-cbdd55773dfa None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:14.656134 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:14.760502 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265556be-4d40-4b8a-a843-cbdd55773dfa None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:14.805443 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6b00f-bd0c-49f8-b74e-abfe9c27522c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:14.806446 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6b00f-bd0c-49f8-b74e-abfe9c27522c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:14.806869 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6b00f-bd0c-49f8-b74e-abfe9c27522c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:14.807314 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6b00f-bd0c-49f8-b74e-abfe9c27522c None admin] PATH_INFO: `/v3/users/a737e413d8744fc49876566629ebfe18` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:14.808578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6b00f-bd0c-49f8-b74e-abfe9c27522c None admin] RBAC: Authorizing `identity:get_user(user_id=a737e413d8744fc49876566629ebfe18)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:14.810191 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6b00f-bd0c-49f8-b74e-abfe9c27522c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:14.811943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 71/148] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:14 2022] GET /identity/v3/users/a737e413d8744fc49876566629ebfe18 => generated 289 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:14.817748 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4ae0c-6ac5-4824-9b4f-041cbc22307e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:14.901212 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:14.963070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4ae0c-6ac5-4824-9b4f-041cbc22307e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:14.975994 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6b00f-bd0c-49f8-b74e-abfe9c27522c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:15.012796 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae30ba3-66c1-4d63-9a20-8c4cd8611a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:15.013446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae30ba3-66c1-4d63-9a20-8c4cd8611a34 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:15.013730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae30ba3-66c1-4d63-9a20-8c4cd8611a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:15.014333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae30ba3-66c1-4d63-9a20-8c4cd8611a34 None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:15.015893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae30ba3-66c1-4d63-9a20-8c4cd8611a34 None admin] RBAC: Authorizing `identity:get_project(project_id=19b75bd757df4c8cac09dd67b1f0ca8d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:15.017351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae30ba3-66c1-4d63-9a20-8c4cd8611a34 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:15.020008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 78/149] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:14 2022] GET /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:15.026492 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae30ba3-66c1-4d63-9a20-8c4cd8611a34 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:15.058232 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:15.096546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6b00f-bd0c-49f8-b74e-abfe9c27522c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:15.134612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb2600-83f8-46f7-9e78-5c9e165cbcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:15.136116 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb2600-83f8-46f7-9e78-5c9e165cbcde None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:15.136854 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb2600-83f8-46f7-9e78-5c9e165cbcde None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:15.137350 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb2600-83f8-46f7-9e78-5c9e165cbcde None admin] PATH_INFO: `/v3/roles/member` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:15.154353 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb2600-83f8-46f7-9e78-5c9e165cbcde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:15.160129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-23eb2600-83f8-46f7-9e78-5c9e165cbcde None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:15.160129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:15.160129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:15.160129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:15.160129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:15.160129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:15.160129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:15.160129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:15.160129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:15.160129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:15.160129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:15.160129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:15.160129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:15.160129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:15.160129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:15.160129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:15.160979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:15.160979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:15.160979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:15.160979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:15.160979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:15.160979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:15.160979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:15.160979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:15.160979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:15.160979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:15.160979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:15.160979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:15.160979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:15.160979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:15.160979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:15.160979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:15.161770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:15.161770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:15.161770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:15.161770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:15.161770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:15.161770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:15.161770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:15.161770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:15.161770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:15.161770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:15.161770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:15.161770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:15.164720 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 72/150] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:15.170981 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb2600-83f8-46f7-9e78-5c9e165cbcde None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:15.207887 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae30ba3-66c1-4d63-9a20-8c4cd8611a34 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:15.236548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:15.267569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34ef82-fd77-4094-aff6-99984b657fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:15.268250 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34ef82-fd77-4094-aff6-99984b657fca None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:15.268534 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34ef82-fd77-4094-aff6-99984b657fca None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:15.268812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34ef82-fd77-4094-aff6-99984b657fca None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d/users/a737e413d8744fc49876566629ebfe18/roles/f792865844594321a8a67b38335cdccb` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:15.270857 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34ef82-fd77-4094-aff6-99984b657fca None admin] RBAC: Authorizing `identity:create_grant(project_id=19b75bd757df4c8cac09dd67b1f0ca8d, user_id=a737e413d8744fc49876566629ebfe18, role_id=f792865844594321a8a67b38335cdccb)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:15.272680 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34ef82-fd77-4094-aff6-99984b657fca None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:15.295411 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb2600-83f8-46f7-9e78-5c9e165cbcde None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:15.326359 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:15.333668 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361bfe8-6dc7-42da-b86c-40a29ce6775a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:15.334323 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361bfe8-6dc7-42da-b86c-40a29ce6775a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:15.334634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361bfe8-6dc7-42da-b86c-40a29ce6775a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:15.334896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361bfe8-6dc7-42da-b86c-40a29ce6775a None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:15.335635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361bfe8-6dc7-42da-b86c-40a29ce6775a None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:15.335953 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361bfe8-6dc7-42da-b86c-40a29ce6775a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:15.337651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361bfe8-6dc7-42da-b86c-40a29ce6775a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:15.346678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 73/151] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:15.352904 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361bfe8-6dc7-42da-b86c-40a29ce6775a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:15.383213 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 79/152] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:45:15 2022] PUT /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d/users/a737e413d8744fc49876566629ebfe18/roles/f792865844594321a8a67b38335cdccb => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 11:45:15.467886 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361bfe8-6dc7-42da-b86c-40a29ce6775a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:15.514893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54f3e9-a1e7-45bf-9a63-dd66f0fee63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:15.515628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54f3e9-a1e7-45bf-9a63-dd66f0fee63b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:15.515923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54f3e9-a1e7-45bf-9a63-dd66f0fee63b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:15.516205 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54f3e9-a1e7-45bf-9a63-dd66f0fee63b None admin] PATH_INFO: `/v3/users/d56b1b366e684a00aa34e438baa482b8` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:15.517727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54f3e9-a1e7-45bf-9a63-dd66f0fee63b None admin] RBAC: Authorizing `identity:get_user(user_id=d56b1b366e684a00aa34e438baa482b8)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:15.519343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54f3e9-a1e7-45bf-9a63-dd66f0fee63b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:15.521341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 74/153] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:15 2022] GET /identity/v3/users/d56b1b366e684a00aa34e438baa482b8 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:15.526986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34ef82-fd77-4094-aff6-99984b657fca None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:15.664531 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34ef82-fd77-4094-aff6-99984b657fca None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:15.704460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9ad30-76a5-4f8a-a688-20152fef077b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:15.705446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9ad30-76a5-4f8a-a688-20152fef077b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:15.706051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9ad30-76a5-4f8a-a688-20152fef077b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:15.706605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9ad30-76a5-4f8a-a688-20152fef077b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:15.762294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:15.787376 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9ad30-76a5-4f8a-a688-20152fef077b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:15.789294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9ad30-76a5-4f8a-a688-20152fef077b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:15.919693 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 80/154] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:15.925786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54f3e9-a1e7-45bf-9a63-dd66f0fee63b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:16.017085 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:16.067747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54f3e9-a1e7-45bf-9a63-dd66f0fee63b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:16.109397 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943ec21-578c-43e4-a40b-95453dc226e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:16.110062 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943ec21-578c-43e4-a40b-95453dc226e4 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:16.110349 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943ec21-578c-43e4-a40b-95453dc226e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:16.110631 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943ec21-578c-43e4-a40b-95453dc226e4 None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:16.111792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1943ec21-578c-43e4-a40b-95453dc226e4 None admin] RBAC: Authorizing `identity:get_project(project_id=19b75bd757df4c8cac09dd67b1f0ca8d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:16.113244 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1943ec21-578c-43e4-a40b-95453dc226e4 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:16.114938 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 75/155] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:15 2022] GET /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:16.120930 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9ad30-76a5-4f8a-a688-20152fef077b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:16.215888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:16.251802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9ad30-76a5-4f8a-a688-20152fef077b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:16.294002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd6da7-d6a3-4ca2-abb6-1b0d774b2024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:16.294551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd6da7-d6a3-4ca2-abb6-1b0d774b2024 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:16.294835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd6da7-d6a3-4ca2-abb6-1b0d774b2024 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:16.295114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd6da7-d6a3-4ca2-abb6-1b0d774b2024 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:16.296464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd6da7-d6a3-4ca2-abb6-1b0d774b2024 None admin] RBAC: Adding query filter params (role.id=d96ee0e293244ae6a04e8ce652aea68e, scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d, user.id=d56b1b366e684a00aa34e438baa482b8) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:16.296784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd6da7-d6a3-4ca2-abb6-1b0d774b2024 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:16.298255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd6da7-d6a3-4ca2-abb6-1b0d774b2024 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:16.305092 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 81/156] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:45:16 2022] GET /identity/v3/role_assignments?user.id=d56b1b366e684a00aa34e438baa482b8&scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d&role.id=d96ee0e293244ae6a04e8ce652aea68e => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:16.492614 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac37f1-67ed-4974-8bcf-90c0568a1dff None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:16.493354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac37f1-67ed-4974-8bcf-90c0568a1dff None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:16.493844 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac37f1-67ed-4974-8bcf-90c0568a1dff None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:16.495427 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 76/157] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:16.501538 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb84f0-36aa-4cf4-82a0-d9733da9fcf1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:16.502064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb84f0-36aa-4cf4-82a0-d9733da9fcf1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:16.502137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb84f0-36aa-4cf4-82a0-d9733da9fcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:16.527019 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-5dcb84f0-36aa-4cf4-82a0-d9733da9fcf1 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:16.528236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5dcb84f0-36aa-4cf4-82a0-d9733da9fcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:16.629519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 82/158] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:16.884662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d37018-1c78-44a4-a099-984a31b7f46f None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:16.884946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d37018-1c78-44a4-a099-984a31b7f46f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:16.885156 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d37018-1c78-44a4-a099-984a31b7f46f None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:16.886559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 77/159] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:16.892674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5279722-12a2-49dc-a241-ff630f4f3f7f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:16.892970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5279722-12a2-49dc-a241-ff630f4f3f7f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:16.893255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5279722-12a2-49dc-a241-ff630f4f3f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:16.916681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a5279722-12a2-49dc-a241-ff630f4f3f7f None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:16.917904 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a5279722-12a2-49dc-a241-ff630f4f3f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:16.995211 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:17.000841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 83/160] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:17.186534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6dd75-8182-4fce-9963-cb8372c3f70d None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:17.187459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6dd75-8182-4fce-9963-cb8372c3f70d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:17.187957 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6dd75-8182-4fce-9963-cb8372c3f70d None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:17.189537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 78/161] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:17.195215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f62c9-6f74-46bd-9487-32cde9272d82 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:17.195510 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f62c9-6f74-46bd-9487-32cde9272d82 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:17.195796 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f62c9-6f74-46bd-9487-32cde9272d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:17.215547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4c8f62c9-6f74-46bd-9487-32cde9272d82 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:17.216747 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4c8f62c9-6f74-46bd-9487-32cde9272d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:17.303122 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 84/162] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:17.343696 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6dd75-8182-4fce-9963-cb8372c3f70d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:17.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f62c9-6f74-46bd-9487-32cde9272d82 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:17.531815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6dd75-8182-4fce-9963-cb8372c3f70d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:17.594207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:17.606820 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f62c9-6f74-46bd-9487-32cde9272d82 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:17.611882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa8353-ba73-470d-ac32-8245827e5a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:17.612588 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa8353-ba73-470d-ac32-8245827e5a1c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:17.612873 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa8353-ba73-470d-ac32-8245827e5a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:17.613148 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa8353-ba73-470d-ac32-8245827e5a1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:17.643662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa8353-ba73-470d-ac32-8245827e5a1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:17.648534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-e4fa8353-ba73-470d-ac32-8245827e5a1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:45:17.648534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:17.648534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:17.648534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:17.648534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:17.648534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:17.648534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:17.648534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:17.648534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:17.648534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:17.648534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:17.648534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:17.648534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:17.648534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:17.648534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:17.648534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:17.649479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:17.649479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:17.649479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:17.649479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:17.649479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:17.649479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:17.649479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:17.649479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:17.649479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:17.649479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:17.649479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:17.649479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:17.649479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:17.649479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:17.649479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:17.649479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:17.650768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:17.650768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:17.650768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:17.650768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:17.650768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:17.650768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:17.650768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:17.650768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:17.650768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:17.650768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:17.650768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:45:17.650768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:17.650768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 79/163] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:45:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:17.656882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa8353-ba73-470d-ac32-8245827e5a1c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:17.670714 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:17.682557 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201b2fd-a81a-4b5c-af8c-ebda959435d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:17.684815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201b2fd-a81a-4b5c-af8c-ebda959435d3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:17.684815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201b2fd-a81a-4b5c-af8c-ebda959435d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:17.684815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201b2fd-a81a-4b5c-af8c-ebda959435d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:17.702372 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201b2fd-a81a-4b5c-af8c-ebda959435d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:17.707803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-d201b2fd-a81a-4b5c-af8c-ebda959435d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:17.707803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:17.707803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:17.707803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:17.707803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:17.707803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:17.707803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:17.707803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:17.707803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:17.707803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:17.707803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:17.707803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:17.707803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:17.707803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:17.707803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:17.707803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:17.708681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:17.708681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:17.708681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:17.708681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:17.708681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:17.708681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:17.708681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:17.708681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:17.708681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:17.708681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:17.708681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:17.708681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:17.708681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:17.708681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:17.708681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:17.708681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:17.709475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:17.709475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:17.709475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:17.709475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:17.709475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:17.709475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:17.709475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:17.709475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:17.709475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:17.709475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:17.709475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:17.709475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:17.711868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 85/164] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:45:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:45:17.722089 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201b2fd-a81a-4b5c-af8c-ebda959435d3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:17.867922 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201b2fd-a81a-4b5c-af8c-ebda959435d3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:17.881830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa8353-ba73-470d-ac32-8245827e5a1c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:17.954575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bcd173-a6f7-4f83-8ce9-b7902d69650f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:17.955201 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcd173-a6f7-4f83-8ce9-b7902d69650f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:17.955491 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcd173-a6f7-4f83-8ce9-b7902d69650f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:17.955768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcd173-a6f7-4f83-8ce9-b7902d69650f None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:17.956441 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bcd173-a6f7-4f83-8ce9-b7902d69650f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:17.956806 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bcd173-a6f7-4f83-8ce9-b7902d69650f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:17.958246 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bcd173-a6f7-4f83-8ce9-b7902d69650f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:17.965094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aeb2eb-744b-4164-b36b-fc0146e4495d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:17.965094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aeb2eb-744b-4164-b36b-fc0146e4495d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:17.965094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aeb2eb-744b-4164-b36b-fc0146e4495d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:17.965094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aeb2eb-744b-4164-b36b-fc0146e4495d None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:17.965452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aeb2eb-744b-4164-b36b-fc0146e4495d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:17.965452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aeb2eb-744b-4164-b36b-fc0146e4495d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:17.965452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aeb2eb-744b-4164-b36b-fc0146e4495d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:17.975942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 86/165] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:45:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:17.976778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bcd173-a6f7-4f83-8ce9-b7902d69650f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:17.979485 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 80/166] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:17.982812 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aeb2eb-744b-4164-b36b-fc0146e4495d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:18.079283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:18.097050 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:18.183965 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aeb2eb-744b-4164-b36b-fc0146e4495d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:18.202583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bcd173-a6f7-4f83-8ce9-b7902d69650f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:18.261232 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b393479-d2db-4573-a041-d8df968ad4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:18.261977 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b393479-d2db-4573-a041-d8df968ad4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:18.262270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b393479-d2db-4573-a041-d8df968ad4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:18.262553 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b393479-d2db-4573-a041-d8df968ad4f5 None admin] PATH_INFO: `/v3/users/d56b1b366e684a00aa34e438baa482b8` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:18.266552 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b393479-d2db-4573-a041-d8df968ad4f5 None admin] RBAC: Authorizing `identity:get_user(user_id=d56b1b366e684a00aa34e438baa482b8)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:18.268151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b393479-d2db-4573-a041-d8df968ad4f5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:18.269832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 81/167] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:17 2022] GET /identity/v3/users/d56b1b366e684a00aa34e438baa482b8 => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:18.270623 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b393479-d2db-4573-a041-d8df968ad4f5 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:18.275915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba8490-0f23-448f-99eb-6562a5e743be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:18.276561 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba8490-0f23-448f-99eb-6562a5e743be None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:18.276843 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba8490-0f23-448f-99eb-6562a5e743be None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:18.277115 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba8490-0f23-448f-99eb-6562a5e743be None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:18.302987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba8490-0f23-448f-99eb-6562a5e743be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:18.308315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-4eba8490-0f23-448f-99eb-6562a5e743be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:18.308315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:18.308315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:18.308315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:18.308315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:18.308315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:18.308315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:18.308315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:18.308315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:18.308315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:18.308315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:18.308315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:18.308315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:18.308315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:18.308315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:18.308315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:18.309192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:18.309192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:18.309192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:18.309192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:18.309192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:18.309192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:18.309192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:18.309192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:18.309192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:18.309192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:18.309192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:18.309192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:18.309192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:18.309192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:18.309192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:18.309192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:18.310419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:18.310419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:18.310419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:18.310419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:18.310419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:18.310419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:18.310419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:18.310419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:18.310419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:18.310419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:18.310419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:18.310419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:18.310419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 87/168] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:18.311149 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba8490-0f23-448f-99eb-6562a5e743be None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:18.458850 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b393479-d2db-4573-a041-d8df968ad4f5 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:18.483167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:18.502815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a2c14-7cab-4321-b7ec-321c4bb93cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:18.503584 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a2c14-7cab-4321-b7ec-321c4bb93cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:18.513197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a2c14-7cab-4321-b7ec-321c4bb93cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:18.513197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a2c14-7cab-4321-b7ec-321c4bb93cf3 None admin] PATH_INFO: `/v3/users/6c2df9ec255e48ec9c7b4291f711f0cd` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:18.513197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a2c14-7cab-4321-b7ec-321c4bb93cf3 None admin] RBAC: Authorizing `identity:get_user(user_id=6c2df9ec255e48ec9c7b4291f711f0cd)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:18.513197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a2c14-7cab-4321-b7ec-321c4bb93cf3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:18.513197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 82/169] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:18 2022] GET /identity/v3/users/6c2df9ec255e48ec9c7b4291f711f0cd => generated 247 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:18.513532 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a2c14-7cab-4321-b7ec-321c4bb93cf3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:18.564830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba8490-0f23-448f-99eb-6562a5e743be None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:18.578693 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:18.617906 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998c7db-472e-4143-a45f-8ab09f0a0984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:18.617906 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998c7db-472e-4143-a45f-8ab09f0a0984 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:18.617906 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998c7db-472e-4143-a45f-8ab09f0a0984 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:18.617906 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998c7db-472e-4143-a45f-8ab09f0a0984 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:18.699211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a2c14-7cab-4321-b7ec-321c4bb93cf3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:18.758073 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ad223-94ef-4f0c-91ef-0a8ee552aa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:18.758612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ad223-94ef-4f0c-91ef-0a8ee552aa79 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:18.758896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ad223-94ef-4f0c-91ef-0a8ee552aa79 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:18.759172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ad223-94ef-4f0c-91ef-0a8ee552aa79 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:18.759845 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ad223-94ef-4f0c-91ef-0a8ee552aa79 None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:18.760150 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ad223-94ef-4f0c-91ef-0a8ee552aa79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:18.761530 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ad223-94ef-4f0c-91ef-0a8ee552aa79 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:18.762915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998c7db-472e-4143-a45f-8ab09f0a0984 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:18.764454 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998c7db-472e-4143-a45f-8ab09f0a0984 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:18.781080 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 83/170] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:18.781956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ad223-94ef-4f0c-91ef-0a8ee552aa79 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:18.895341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:18.968123 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 88/171] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:18.969002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998c7db-472e-4143-a45f-8ab09f0a0984 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:18.993496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:19.005206 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ad223-94ef-4f0c-91ef-0a8ee552aa79 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:19.086560 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998c7db-472e-4143-a45f-8ab09f0a0984 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:19.118635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e0706-496e-4642-a3f7-981ecd731d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:19.119264 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e0706-496e-4642-a3f7-981ecd731d3d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:19.119546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e0706-496e-4642-a3f7-981ecd731d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:19.119826 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e0706-496e-4642-a3f7-981ecd731d3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:19.123926 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06051242-6742-40e8-9bab-36bc381f4f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:19.125183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06051242-6742-40e8-9bab-36bc381f4f9a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:19.125471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06051242-6742-40e8-9bab-36bc381f4f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:19.125737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06051242-6742-40e8-9bab-36bc381f4f9a None admin] PATH_INFO: `/v3/users/d56b1b366e684a00aa34e438baa482b8` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:19.126918 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06051242-6742-40e8-9bab-36bc381f4f9a None admin] RBAC: Authorizing `identity:get_user(user_id=d56b1b366e684a00aa34e438baa482b8)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:19.128370 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06051242-6742-40e8-9bab-36bc381f4f9a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:19.129984 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 89/172] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:18 2022] GET /identity/v3/users/d56b1b366e684a00aa34e438baa482b8 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:19.131145 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51209f1-42f1-428e-9edf-17f069b3f45f None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:19.131436 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51209f1-42f1-428e-9edf-17f069b3f45f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:19.131708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51209f1-42f1-428e-9edf-17f069b3f45f None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:19.132893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 90/173] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:19.133620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51209f1-42f1-428e-9edf-17f069b3f45f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:19.198671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e0706-496e-4642-a3f7-981ecd731d3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:19.200366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e0706-496e-4642-a3f7-981ecd731d3d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:19.252844 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51209f1-42f1-428e-9edf-17f069b3f45f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:19.255880 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:19.289094 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa8ecb-e05c-449b-b61b-915a9496edd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:19.289732 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa8ecb-e05c-449b-b61b-915a9496edd3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:19.290037 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa8ecb-e05c-449b-b61b-915a9496edd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:19.290314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa8ecb-e05c-449b-b61b-915a9496edd3 None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:19.291730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa8ecb-e05c-449b-b61b-915a9496edd3 None admin] RBAC: Authorizing `identity:get_project(project_id=19b75bd757df4c8cac09dd67b1f0ca8d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:19.293599 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa8ecb-e05c-449b-b61b-915a9496edd3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:19.295717 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 91/174] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:19 2022] GET /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:19.297208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6dba92-a8a7-4ce6-926a-9b2a03aaf130 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:19.297486 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6dba92-a8a7-4ce6-926a-9b2a03aaf130 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:19.297760 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6dba92-a8a7-4ce6-926a-9b2a03aaf130 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:19.307674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:19.309478 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 92/175] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:19 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:19.311165 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6dba92-a8a7-4ce6-926a-9b2a03aaf130 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:19.329939 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 84/176] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:19.329939 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294dea10-dba4-4abb-81cb-58db09b79d98 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:19.329939 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294dea10-dba4-4abb-81cb-58db09b79d98 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:19.329939 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294dea10-dba4-4abb-81cb-58db09b79d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:19.350279 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-294dea10-dba4-4abb-81cb-58db09b79d98 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:19.354833 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-294dea10-dba4-4abb-81cb-58db09b79d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:19.433883 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 85/177] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:19.433883 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294dea10-dba4-4abb-81cb-58db09b79d98 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:19.469741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6dba92-a8a7-4ce6-926a-9b2a03aaf130 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:19.516049 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ab63a-664b-484e-b51a-35656144f0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:19.516773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ab63a-664b-484e-b51a-35656144f0d6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:19.517064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ab63a-664b-484e-b51a-35656144f0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:19.517341 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ab63a-664b-484e-b51a-35656144f0d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:19.527661 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:19.538944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:19.553242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294dea10-dba4-4abb-81cb-58db09b79d98 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:19.590459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5759c2a9-faef-4b1f-9907-c9eb2eb34c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:19.590954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ab63a-664b-484e-b51a-35656144f0d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:19.599974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759c2a9-faef-4b1f-9907-c9eb2eb34c57 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:19.599974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759c2a9-faef-4b1f-9907-c9eb2eb34c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:19.599974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759c2a9-faef-4b1f-9907-c9eb2eb34c57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:19.599974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5759c2a9-faef-4b1f-9907-c9eb2eb34c57 None admin] RBAC: Adding query filter params (role.id=1ab8c505dc344769abc347f66e529527, scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d, user.id=d56b1b366e684a00aa34e438baa482b8) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:19.600371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ab63a-664b-484e-b51a-35656144f0d6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:19.600471 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5759c2a9-faef-4b1f-9907-c9eb2eb34c57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:19.600471 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5759c2a9-faef-4b1f-9907-c9eb2eb34c57 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:19.601575 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 86/178] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:45:19 2022] GET /identity/v3/role_assignments?user.id=d56b1b366e684a00aa34e438baa482b8&scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d&role.id=1ab8c505dc344769abc347f66e529527 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:19.602840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640dccb-8300-4a94-9ba2-23cd492db625 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:19.603131 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640dccb-8300-4a94-9ba2-23cd492db625 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:19.603415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640dccb-8300-4a94-9ba2-23cd492db625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:19.626064 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0640dccb-8300-4a94-9ba2-23cd492db625 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:19.627512 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0640dccb-8300-4a94-9ba2-23cd492db625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:19.722936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 93/179] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:19.727056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ab63a-664b-484e-b51a-35656144f0d6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:19.729647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 87/180] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:19.731855 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896688c-f68f-4efa-b20e-fc07029bb791 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:19.731855 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896688c-f68f-4efa-b20e-fc07029bb791 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:19.731855 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896688c-f68f-4efa-b20e-fc07029bb791 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:19.732911 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 88/181] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:19.739077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896688c-f68f-4efa-b20e-fc07029bb791 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:19.830888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:19.853995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:19.893632 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ab63a-664b-484e-b51a-35656144f0d6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:19.896260 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896688c-f68f-4efa-b20e-fc07029bb791 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:19.936996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b823985-6475-498c-ba7b-dc7c07614a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:19.937818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b823985-6475-498c-ba7b-dc7c07614a5b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:19.938166 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b823985-6475-498c-ba7b-dc7c07614a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:19.938438 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b823985-6475-498c-ba7b-dc7c07614a5b None admin] PATH_INFO: `/v3/projects/b6871445408344a4b618cafb428583ce` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:19.938722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5a772-6465-489d-abbc-e5978719d31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:19.939364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5a772-6465-489d-abbc-e5978719d31c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:19.939555 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b823985-6475-498c-ba7b-dc7c07614a5b None admin] RBAC: Authorizing `identity:get_project(project_id=b6871445408344a4b618cafb428583ce)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:19.939657 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5a772-6465-489d-abbc-e5978719d31c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:19.939911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5a772-6465-489d-abbc-e5978719d31c None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:19.940989 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b823985-6475-498c-ba7b-dc7c07614a5b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:19.941097 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5a772-6465-489d-abbc-e5978719d31c None admin] RBAC: Authorizing `identity:get_project(project_id=19b75bd757df4c8cac09dd67b1f0ca8d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:19.942796 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5a772-6465-489d-abbc-e5978719d31c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:19.942910 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 89/182] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:19 2022] GET /identity/v3/projects/b6871445408344a4b618cafb428583ce => generated 312 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:19.943749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c6326-8465-43a8-b4e3-3ad5777f8353 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:19.944008 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c6326-8465-43a8-b4e3-3ad5777f8353 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:19.944335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c6326-8465-43a8-b4e3-3ad5777f8353 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:19.945957 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 90/183] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:19.946710 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43ecb5-caf0-4652-94a9-19eca5b55a1b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:19.946988 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43ecb5-caf0-4652-94a9-19eca5b55a1b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:19.947256 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43ecb5-caf0-4652-94a9-19eca5b55a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:19.957137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 94/184] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:19 2022] GET /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:19.957137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5a772-6465-489d-abbc-e5978719d31c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:19.966072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-bf43ecb5-caf0-4652-94a9-19eca5b55a1b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:19.967251 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-bf43ecb5-caf0-4652-94a9-19eca5b55a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:20.039395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 91/185] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:20.040534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43ecb5-caf0-4652-94a9-19eca5b55a1b None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:20.063541 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5a772-6465-489d-abbc-e5978719d31c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:20.082896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:20.101525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75932cb8-d673-4a27-83bc-5527337c3f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:20.103020 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75932cb8-d673-4a27-83bc-5527337c3f72 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:20.103369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75932cb8-d673-4a27-83bc-5527337c3f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:20.103654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75932cb8-d673-4a27-83bc-5527337c3f72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:20.112359 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:20.114952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75932cb8-d673-4a27-83bc-5527337c3f72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:20.119878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-75932cb8-d673-4a27-83bc-5527337c3f72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:45:20.119878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:20.119878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:20.119878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:20.119878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:20.119878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:20.119878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:20.119878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:20.119878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:20.119878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:20.119878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:20.119878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:20.119878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:20.119878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:20.119878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:20.119878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:20.126945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:20.126945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:20.126945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:20.126945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:20.126945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:20.126945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:20.126945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:20.126945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:20.126945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:20.126945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:20.126945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:20.126945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:20.126945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:20.126945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:20.126945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:20.126945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:20.127769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:20.127769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:20.127769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:20.127769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:20.127769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:20.127769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:20.127769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:20.127769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:20.127769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:20.127769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:20.127769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:45:20.127769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:20.127769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 95/186] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:45:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:20.127769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75932cb8-d673-4a27-83bc-5527337c3f72 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:20.145453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43ecb5-caf0-4652-94a9-19eca5b55a1b None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:20.181406 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb458d-b37b-4470-8a68-daaf55821da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:20.182083 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb458d-b37b-4470-8a68-daaf55821da4 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:20.182377 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb458d-b37b-4470-8a68-daaf55821da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:20.182607 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb458d-b37b-4470-8a68-daaf55821da4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:20.183444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb458d-b37b-4470-8a68-daaf55821da4 None admin] RBAC: Adding query filter params (role.id=d96ee0e293244ae6a04e8ce652aea68e, scope.project.id=b6871445408344a4b618cafb428583ce, user.id=d56b1b366e684a00aa34e438baa482b8) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:20.183650 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb458d-b37b-4470-8a68-daaf55821da4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:20.185082 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb458d-b37b-4470-8a68-daaf55821da4 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:20.191138 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 92/187] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:45:20 2022] GET /identity/v3/role_assignments?user.id=d56b1b366e684a00aa34e438baa482b8&scope.project.id=b6871445408344a4b618cafb428583ce&role.id=d96ee0e293244ae6a04e8ce652aea68e => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:20.192102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8404eb-9bea-4f24-8d60-5ccbbcc077d4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:20.192387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8404eb-9bea-4f24-8d60-5ccbbcc077d4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:20.192663 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8404eb-9bea-4f24-8d60-5ccbbcc077d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:20.213011 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0b8404eb-9bea-4f24-8d60-5ccbbcc077d4 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:20.214262 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0b8404eb-9bea-4f24-8d60-5ccbbcc077d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:20.260494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75932cb8-d673-4a27-83bc-5527337c3f72 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:20.301886 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 93/188] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:20.301886 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8404eb-9bea-4f24-8d60-5ccbbcc077d4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:20.319383 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992745c6-2bea-4270-a38e-a9a342ed715f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:20.319997 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992745c6-2bea-4270-a38e-a9a342ed715f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:20.320281 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992745c6-2bea-4270-a38e-a9a342ed715f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:20.320549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992745c6-2bea-4270-a38e-a9a342ed715f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:20.321317 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992745c6-2bea-4270-a38e-a9a342ed715f None admin] RBAC: Adding query filter params (role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d, user.id=6c2df9ec255e48ec9c7b4291f711f0cd) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:20.321633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992745c6-2bea-4270-a38e-a9a342ed715f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:20.323206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992745c6-2bea-4270-a38e-a9a342ed715f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:20.330823 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 96/189] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:45:20 2022] GET /identity/v3/role_assignments?user.id=6c2df9ec255e48ec9c7b4291f711f0cd&scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:20.338154 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992745c6-2bea-4270-a38e-a9a342ed715f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:20.398489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:20.406395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:20.448858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8404eb-9bea-4f24-8d60-5ccbbcc077d4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:20.487801 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992745c6-2bea-4270-a38e-a9a342ed715f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:20.500092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871d974-e3ed-49ea-9cd7-2fce41bc7d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:20.501718 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871d974-e3ed-49ea-9cd7-2fce41bc7d9a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:20.502250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871d974-e3ed-49ea-9cd7-2fce41bc7d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:20.502745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871d974-e3ed-49ea-9cd7-2fce41bc7d9a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:20.517604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f871d974-e3ed-49ea-9cd7-2fce41bc7d9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:20.526580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-f871d974-e3ed-49ea-9cd7-2fce41bc7d9a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 11:45:20.526580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:20.526580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:20.526580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:20.526580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:20.526580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:20.526580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:20.526580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:20.526580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:20.526580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:20.526580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:20.526580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:20.526580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:20.526580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:20.526580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:20.526580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:20.527558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:20.527558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:20.527558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:20.527558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:20.527558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:20.527558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:20.527558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:20.527558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:20.527558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:20.527558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:20.527558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:20.527558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:20.527558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:20.527558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:20.527558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:20.527558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:20.528336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:20.528336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:20.528336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:20.528336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:20.528336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:20.528336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:20.528336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:20.528336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:20.528336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:20.528336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:20.528336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 11:45:20.528336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:20.530913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 94/190] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:20.531994 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871d974-e3ed-49ea-9cd7-2fce41bc7d9a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:20.534852 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa06637-fe34-4071-9bae-6c61d7d8e393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:20.535490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa06637-fe34-4071-9bae-6c61d7d8e393 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:20.536104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa06637-fe34-4071-9bae-6c61d7d8e393 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:20.536396 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa06637-fe34-4071-9bae-6c61d7d8e393 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:20.537087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa06637-fe34-4071-9bae-6c61d7d8e393 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:20.537418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa06637-fe34-4071-9bae-6c61d7d8e393 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:20.539899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa06637-fe34-4071-9bae-6c61d7d8e393 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:20.553786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 97/191] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:20.554932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa06637-fe34-4071-9bae-6c61d7d8e393 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:20.665447 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871d974-e3ed-49ea-9cd7-2fce41bc7d9a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:20.674958 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa06637-fe34-4071-9bae-6c61d7d8e393 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:20.676751 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:20.691733 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:20.701631 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27647d9-af85-44d9-b2c6-ec5fb21998c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:20.702316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27647d9-af85-44d9-b2c6-ec5fb21998c6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:20.702598 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27647d9-af85-44d9-b2c6-ec5fb21998c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:20.702862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27647d9-af85-44d9-b2c6-ec5fb21998c6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:20.709044 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b1f1c-25d3-4ac3-a4a3-ff89ec2cbbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:20.709647 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b1f1c-25d3-4ac3-a4a3-ff89ec2cbbe6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:20.709965 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b1f1c-25d3-4ac3-a4a3-ff89ec2cbbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:20.710249 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b1f1c-25d3-4ac3-a4a3-ff89ec2cbbe6 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:20.710916 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b1f1c-25d3-4ac3-a4a3-ff89ec2cbbe6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:20.711226 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b1f1c-25d3-4ac3-a4a3-ff89ec2cbbe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:20.712500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27647d9-af85-44d9-b2c6-ec5fb21998c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:20.712613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b1f1c-25d3-4ac3-a4a3-ff89ec2cbbe6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:20.717341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d27647d9-af85-44d9-b2c6-ec5fb21998c6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:20.717341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:20.717341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:20.717341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:20.717341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:20.717341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:20.717341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:20.717341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:20.717341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:20.717341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:20.717341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:20.717341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:20.717341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:20.717341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:20.717341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:20.717341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:20.721881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:20.721881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:20.721881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:20.721881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:20.721881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:20.721881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:20.721881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:20.721881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:20.721881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:20.721881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:20.721881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:20.721881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:20.721881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:20.721881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:20.721881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:20.721881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:20.722708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:20.722708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:20.722708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:20.722708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:20.722708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:20.722708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:20.722708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:20.722708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:20.722708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:20.722708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:20.722708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:20.722708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:20.722708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 95/192] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:45:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:20.722708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27647d9-af85-44d9-b2c6-ec5fb21998c6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:20.730262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 98/193] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:20.730262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b1f1c-25d3-4ac3-a4a3-ff89ec2cbbe6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:20.822836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27647d9-af85-44d9-b2c6-ec5fb21998c6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:20.825820 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b1f1c-25d3-4ac3-a4a3-ff89ec2cbbe6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:20.856226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3e78b-82ff-4732-9062-cc425283ec64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:20.856851 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3e78b-82ff-4732-9062-cc425283ec64 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:20.857126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3e78b-82ff-4732-9062-cc425283ec64 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:20.857397 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3e78b-82ff-4732-9062-cc425283ec64 None admin] PATH_INFO: `/v3/users/d6d10c0a2e0f4e27a799877ea78a13aa` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:20.862944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60734dc8-d095-40b7-912a-1c18351eab44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:20.863578 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60734dc8-d095-40b7-912a-1c18351eab44 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:20.863864 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60734dc8-d095-40b7-912a-1c18351eab44 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:20.864144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60734dc8-d095-40b7-912a-1c18351eab44 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:20.865255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60734dc8-d095-40b7-912a-1c18351eab44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:20.866741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60734dc8-d095-40b7-912a-1c18351eab44 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:20.868292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 99/194] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:45:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:20.869059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60734dc8-d095-40b7-912a-1c18351eab44 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:20.874763 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb3e78b-82ff-4732-9062-cc425283ec64 None admin] RBAC: Authorizing `identity:get_user(user_id=d6d10c0a2e0f4e27a799877ea78a13aa)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:20.876218 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb3e78b-82ff-4732-9062-cc425283ec64 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:20.877873 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 96/195] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:20 2022] GET /identity/v3/users/d6d10c0a2e0f4e27a799877ea78a13aa => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:20.878631 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3e78b-82ff-4732-9062-cc425283ec64 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:20.904239 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:20.920668 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:20.973105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60734dc8-d095-40b7-912a-1c18351eab44 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:20.976273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3e78b-82ff-4732-9062-cc425283ec64 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:21.005674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd60c75-4819-45a8-a244-72481a73c05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:21.006313 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd60c75-4819-45a8-a244-72481a73c05a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:21.006599 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd60c75-4819-45a8-a244-72481a73c05a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:21.006879 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd60c75-4819-45a8-a244-72481a73c05a None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:21.007544 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd60c75-4819-45a8-a244-72481a73c05a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:21.007853 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd60c75-4819-45a8-a244-72481a73c05a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:21.009210 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ad057-4ca3-48b9-8be1-fe3c0ad7b5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:21.009333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd60c75-4819-45a8-a244-72481a73c05a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:21.009838 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ad057-4ca3-48b9-8be1-fe3c0ad7b5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:21.010148 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ad057-4ca3-48b9-8be1-fe3c0ad7b5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:21.010430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ad057-4ca3-48b9-8be1-fe3c0ad7b5d9 None admin] PATH_INFO: `/v3/users/a737e413d8744fc49876566629ebfe18` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:21.011551 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ad057-4ca3-48b9-8be1-fe3c0ad7b5d9 None admin] RBAC: Authorizing `identity:get_user(user_id=a737e413d8744fc49876566629ebfe18)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:21.013000 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ad057-4ca3-48b9-8be1-fe3c0ad7b5d9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:21.014708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 97/196] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:20 2022] GET /identity/v3/users/a737e413d8744fc49876566629ebfe18 => generated 289 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:21.017105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 100/197] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:45:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:21.018701 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ad057-4ca3-48b9-8be1-fe3c0ad7b5d9 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:21.019814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd60c75-4819-45a8-a244-72481a73c05a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:21.115387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:21.121636 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd60c75-4819-45a8-a244-72481a73c05a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:21.122820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ad057-4ca3-48b9-8be1-fe3c0ad7b5d9 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:21.133228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:21.154441 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca39318-fd40-4345-9406-b9ba88ff525f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:21.155583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca39318-fd40-4345-9406-b9ba88ff525f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:21.155864 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca39318-fd40-4345-9406-b9ba88ff525f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:21.155955 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e20883-a67e-40c2-9de3-d3305dfcdb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:21.156148 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca39318-fd40-4345-9406-b9ba88ff525f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:21.156561 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e20883-a67e-40c2-9de3-d3305dfcdb53 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:21.156846 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e20883-a67e-40c2-9de3-d3305dfcdb53 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:21.157129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e20883-a67e-40c2-9de3-d3305dfcdb53 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:21.157780 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e20883-a67e-40c2-9de3-d3305dfcdb53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:21.159248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e20883-a67e-40c2-9de3-d3305dfcdb53 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:21.223724 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca39318-fd40-4345-9406-b9ba88ff525f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:21.225300 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca39318-fd40-4345-9406-b9ba88ff525f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:21.259867 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 98/198] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 14 11:45:21 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:45:21.260675 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e20883-a67e-40c2-9de3-d3305dfcdb53 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:21.341320 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:21.351219 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 101/199] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:21.351219 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca39318-fd40-4345-9406-b9ba88ff525f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:21.379177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e20883-a67e-40c2-9de3-d3305dfcdb53 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:21.418527 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c8d36-d5fe-46ac-a6b4-3a9e22e6f699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:21.419193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c8d36-d5fe-46ac-a6b4-3a9e22e6f699 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:21.419576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c8d36-d5fe-46ac-a6b4-3a9e22e6f699 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:21.419745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c8d36-d5fe-46ac-a6b4-3a9e22e6f699 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:21.459900 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:21.472551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca39318-fd40-4345-9406-b9ba88ff525f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:21.496584 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c8d36-d5fe-46ac-a6b4-3a9e22e6f699 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:21.498784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c8d36-d5fe-46ac-a6b4-3a9e22e6f699 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:21.508936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd930a5b-6690-4db9-928f-4aa907f7c8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:21.513312 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd930a5b-6690-4db9-928f-4aa907f7c8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:21.513312 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd930a5b-6690-4db9-928f-4aa907f7c8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:21.513312 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd930a5b-6690-4db9-928f-4aa907f7c8d1 None admin] PATH_INFO: `/v3/users/d6d10c0a2e0f4e27a799877ea78a13aa` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:21.513312 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd930a5b-6690-4db9-928f-4aa907f7c8d1 None admin] RBAC: Authorizing `identity:get_user(user_id=d6d10c0a2e0f4e27a799877ea78a13aa)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:21.513312 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd930a5b-6690-4db9-928f-4aa907f7c8d1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:21.514270 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 102/200] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:21 2022] GET /identity/v3/users/d6d10c0a2e0f4e27a799877ea78a13aa => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:21.515568 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd930a5b-6690-4db9-928f-4aa907f7c8d1 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:21.596850 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:21.630888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 99/201] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:21.631733 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c8d36-d5fe-46ac-a6b4-3a9e22e6f699 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:21.641448 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd930a5b-6690-4db9-928f-4aa907f7c8d1 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:21.681910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ce23a-ecf3-4c64-b8ab-3cdc3f4816ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:21.682710 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ce23a-ecf3-4c64-b8ab-3cdc3f4816ab None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:21.683115 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ce23a-ecf3-4c64-b8ab-3cdc3f4816ab None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:21.683528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ce23a-ecf3-4c64-b8ab-3cdc3f4816ab None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:21.684781 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ce23a-ecf3-4c64-b8ab-3cdc3f4816ab None admin] RBAC: Authorizing `identity:get_project(project_id=2164ed8287f243bc9a4a5347d4dc0b1d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:21.686988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ce23a-ecf3-4c64-b8ab-3cdc3f4816ab None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:21.688912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 103/202] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:21 2022] GET /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:21.694339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ce23a-ecf3-4c64-b8ab-3cdc3f4816ab None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:21.711066 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:21.746673 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c8d36-d5fe-46ac-a6b4-3a9e22e6f699 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:21.784172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defde62a-d16a-4fe4-9d37-25c67d2cc488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:21.784786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defde62a-d16a-4fe4-9d37-25c67d2cc488 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:21.785069 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defde62a-d16a-4fe4-9d37-25c67d2cc488 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:21.785338 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defde62a-d16a-4fe4-9d37-25c67d2cc488 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:21.809548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ce23a-ecf3-4c64-b8ab-3cdc3f4816ab None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:21.844466 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:21.851389 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f65d22-8aad-4564-b706-5ad3a37c284f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:21.852271 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f65d22-8aad-4564-b706-5ad3a37c284f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:21.852719 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f65d22-8aad-4564-b706-5ad3a37c284f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:21.855523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f65d22-8aad-4564-b706-5ad3a37c284f None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:21.855523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f65d22-8aad-4564-b706-5ad3a37c284f None admin] RBAC: Authorizing `identity:get_project(project_id=19b75bd757df4c8cac09dd67b1f0ca8d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:21.856218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f65d22-8aad-4564-b706-5ad3a37c284f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:21.863503 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 104/203] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:21 2022] GET /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:21.864457 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f65d22-8aad-4564-b706-5ad3a37c284f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:21.866716 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defde62a-d16a-4fe4-9d37-25c67d2cc488 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:21.868187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defde62a-d16a-4fe4-9d37-25c67d2cc488 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:21.939202 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:21.975592 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 100/204] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:21.976647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defde62a-d16a-4fe4-9d37-25c67d2cc488 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:22.023637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f65d22-8aad-4564-b706-5ad3a37c284f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:22.061821 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5b2c3-7bad-4395-ad6b-61f49d16106f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:22.062579 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5b2c3-7bad-4395-ad6b-61f49d16106f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:22.062866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5b2c3-7bad-4395-ad6b-61f49d16106f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:22.063143 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5b2c3-7bad-4395-ad6b-61f49d16106f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:22.064035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5b2c3-7bad-4395-ad6b-61f49d16106f None admin] RBAC: Adding query filter params (role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d, user.id=d6d10c0a2e0f4e27a799877ea78a13aa) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:22.064346 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5b2c3-7bad-4395-ad6b-61f49d16106f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:22.065788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5b2c3-7bad-4395-ad6b-61f49d16106f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:22.071777 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 105/205] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:45:21 2022] GET /identity/v3/role_assignments?user.id=d6d10c0a2e0f4e27a799877ea78a13aa&scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:22.078915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5b2c3-7bad-4395-ad6b-61f49d16106f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:22.120104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defde62a-d16a-4fe4-9d37-25c67d2cc488 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:22.151977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:22.165824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0e626-e5d7-4781-9dfc-459a312caf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:22.166534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0e626-e5d7-4781-9dfc-459a312caf4e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:22.166825 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0e626-e5d7-4781-9dfc-459a312caf4e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:22.167103 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0e626-e5d7-4781-9dfc-459a312caf4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:22.167853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0e626-e5d7-4781-9dfc-459a312caf4e None admin] RBAC: Adding query filter params (role.id=f792865844594321a8a67b38335cdccb, scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d, user.id=a737e413d8744fc49876566629ebfe18) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:22.168299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0e626-e5d7-4781-9dfc-459a312caf4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:22.178950 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0e626-e5d7-4781-9dfc-459a312caf4e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:22.193181 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 101/206] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:45:21 2022] GET /identity/v3/role_assignments?user.id=a737e413d8744fc49876566629ebfe18&scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d&role.id=f792865844594321a8a67b38335cdccb => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:22.237361 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5b2c3-7bad-4395-ad6b-61f49d16106f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:22.297565 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69896a1d-e3a8-4853-9269-8f4fd498b2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:22.298191 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69896a1d-e3a8-4853-9269-8f4fd498b2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:22.298476 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69896a1d-e3a8-4853-9269-8f4fd498b2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:22.298751 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69896a1d-e3a8-4853-9269-8f4fd498b2b9 None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:22.300559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69896a1d-e3a8-4853-9269-8f4fd498b2b9 None admin] RBAC: Authorizing `identity:get_project(project_id=2164ed8287f243bc9a4a5347d4dc0b1d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:22.302120 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69896a1d-e3a8-4853-9269-8f4fd498b2b9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:22.304104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 106/207] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:22 2022] GET /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:22.310419 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0e626-e5d7-4781-9dfc-459a312caf4e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:22.345150 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:22.432991 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0e626-e5d7-4781-9dfc-459a312caf4e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:22.478774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f3398-81fc-4d18-ba06-1f362052dfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:22.479381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f3398-81fc-4d18-ba06-1f362052dfdb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:22.479664 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f3398-81fc-4d18-ba06-1f362052dfdb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:22.479944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f3398-81fc-4d18-ba06-1f362052dfdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:22.480676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f3398-81fc-4d18-ba06-1f362052dfdb None admin] RBAC: Adding query filter params (role.id=1ab8c505dc344769abc347f66e529527, scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d, user.id=d6d10c0a2e0f4e27a799877ea78a13aa) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:22.481052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f3398-81fc-4d18-ba06-1f362052dfdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:22.482573 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f3398-81fc-4d18-ba06-1f362052dfdb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:22.494028 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 102/208] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:45:22 2022] GET /identity/v3/role_assignments?user.id=d6d10c0a2e0f4e27a799877ea78a13aa&scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d&role.id=1ab8c505dc344769abc347f66e529527 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:23.681953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82993f27-3f4e-4c8e-8b42-ae950dea6381 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:23.682511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82993f27-3f4e-4c8e-8b42-ae950dea6381 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:23.682511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82993f27-3f4e-4c8e-8b42-ae950dea6381 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:23.683801 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b414f9-d29f-4684-809a-1252d960867f None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:23.684174 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 107/209] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:23.684587 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b414f9-d29f-4684-809a-1252d960867f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:23.685073 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b414f9-d29f-4684-809a-1252d960867f None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:23.687560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 103/210] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:23.690124 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912e552-4d81-4bd0-a92a-b66d192af618 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:23.690419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912e552-4d81-4bd0-a92a-b66d192af618 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:23.690699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912e552-4d81-4bd0-a92a-b66d192af618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:23.692170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926998c-2991-4264-a272-ee095afc354a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:23.692459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926998c-2991-4264-a272-ee095afc354a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:23.692735 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926998c-2991-4264-a272-ee095afc354a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:23.709512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-2912e552-4d81-4bd0-a92a-b66d192af618 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:23.710359 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-5926998c-2991-4264-a272-ee095afc354a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:23.710735 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2912e552-4d81-4bd0-a92a-b66d192af618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:23.711518 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5926998c-2991-4264-a272-ee095afc354a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:23.790402 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 104/211] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:45:23.799708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 108/212] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:24.064972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926998c-2991-4264-a272-ee095afc354a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:24.080187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912e552-4d81-4bd0-a92a-b66d192af618 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:24.095040 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:24.151198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:24.176607 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926998c-2991-4264-a272-ee095afc354a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:24.195628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912e552-4d81-4bd0-a92a-b66d192af618 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:24.213563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196aa17-0eae-48d5-a027-4e1c72e2bf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:24.214682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196aa17-0eae-48d5-a027-4e1c72e2bf49 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:24.215247 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196aa17-0eae-48d5-a027-4e1c72e2bf49 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:24.215784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196aa17-0eae-48d5-a027-4e1c72e2bf49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:24.228062 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e196aa17-0eae-48d5-a027-4e1c72e2bf49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:24.233163 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57776d40-1657-479c-a019-13193706cd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:24.234387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-e196aa17-0eae-48d5-a027-4e1c72e2bf49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:24.234387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:24.234387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:24.234387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:24.234387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:24.234387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:24.234387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:24.234387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:24.234387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:24.234387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:24.234387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:24.234387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:24.234387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:24.234387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:24.234387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:24.234387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:24.235253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:24.235253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:24.235253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:24.235253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:24.235253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:24.235253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:24.235253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:24.235253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:24.235253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:24.235253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:24.235253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:24.235253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:24.235253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:24.235253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:24.235253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:24.235253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:24.236058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:24.236058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:24.236058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:24.236058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:24.236058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:24.236058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:24.236058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:24.236058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:24.236058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:24.236058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:24.236058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:24.236058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:24.236670 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57776d40-1657-479c-a019-13193706cd8e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:24.236670 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57776d40-1657-479c-a019-13193706cd8e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:24.236670 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57776d40-1657-479c-a019-13193706cd8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:24.236670 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57776d40-1657-479c-a019-13193706cd8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:24.238222 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57776d40-1657-479c-a019-13193706cd8e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:24.238925 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 105/213] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:24.242987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 109/214] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:45:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:24.247056 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196aa17-0eae-48d5-a027-4e1c72e2bf49 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:24.248710 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57776d40-1657-479c-a019-13193706cd8e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:24.345416 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:24.360554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57776d40-1657-479c-a019-13193706cd8e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:24.365249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196aa17-0eae-48d5-a027-4e1c72e2bf49 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:24.390071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:24.398022 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999574c-1eef-4ffb-8b8d-baebb50efc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:24.398583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999574c-1eef-4ffb-8b8d-baebb50efc23 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:24.398867 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999574c-1eef-4ffb-8b8d-baebb50efc23 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:24.399133 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999574c-1eef-4ffb-8b8d-baebb50efc23 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:24.399818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999574c-1eef-4ffb-8b8d-baebb50efc23 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:24.401458 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999574c-1eef-4ffb-8b8d-baebb50efc23 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:24.407243 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87c769-0927-4941-a111-e68abac889fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:24.408228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87c769-0927-4941-a111-e68abac889fc None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:24.408752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87c769-0927-4941-a111-e68abac889fc None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:24.409231 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87c769-0927-4941-a111-e68abac889fc None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:24.410150 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87c769-0927-4941-a111-e68abac889fc None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:24.410662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87c769-0927-4941-a111-e68abac889fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:24.412254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87c769-0927-4941-a111-e68abac889fc None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:24.422374 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 106/215] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:24.428269 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87c769-0927-4941-a111-e68abac889fc None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:24.507117 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 110/216] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 14 11:45:24 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:45:24.508514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bbb86-bdc1-45fd-bf29-1fca57a96afb None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:24.509021 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bbb86-bdc1-45fd-bf29-1fca57a96afb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:24.509125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bbb86-bdc1-45fd-bf29-1fca57a96afb None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:24.510568 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 111/217] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:24.517027 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0c460-c5e6-4aac-8f0d-ed8f5af4cfea None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:24.517394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0c460-c5e6-4aac-8f0d-ed8f5af4cfea None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:24.517564 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0c460-c5e6-4aac-8f0d-ed8f5af4cfea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:24.538383 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4ba0c460-c5e6-4aac-8f0d-ed8f5af4cfea None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:24.539673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4ba0c460-c5e6-4aac-8f0d-ed8f5af4cfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:24.546177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87c769-0927-4941-a111-e68abac889fc None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:24.591303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc9f80d-3edc-445c-81bf-0cd4b55a567e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:24.591537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9f80d-3edc-445c-81bf-0cd4b55a567e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:24.591617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9f80d-3edc-445c-81bf-0cd4b55a567e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:24.591909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9f80d-3edc-445c-81bf-0cd4b55a567e None admin] PATH_INFO: `/v3/users/5b5c68ce791b4845af27773a417486fd` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:24.608355 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc9f80d-3edc-445c-81bf-0cd4b55a567e None admin] RBAC: Authorizing `identity:get_user(user_id=5b5c68ce791b4845af27773a417486fd)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:24.609779 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc9f80d-3edc-445c-81bf-0cd4b55a567e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:24.611631 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 107/218] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:24 2022] GET /identity/v3/users/5b5c68ce791b4845af27773a417486fd => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:24.617658 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc9f80d-3edc-445c-81bf-0cd4b55a567e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:24.622805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:24.644399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 112/219] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:24.744199 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc9f80d-3edc-445c-81bf-0cd4b55a567e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:24.747295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc198021-ce5b-4704-b5c7-437648ee13c9 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:24.748132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc198021-ce5b-4704-b5c7-437648ee13c9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:24.748615 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc198021-ce5b-4704-b5c7-437648ee13c9 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:24.754880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 113/220] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:45:24.760835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dfc18-7a49-4f4d-a949-8d33d5c7ac09 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:24.761129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dfc18-7a49-4f4d-a949-8d33d5c7ac09 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:24.761413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dfc18-7a49-4f4d-a949-8d33d5c7ac09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:24.782008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-fa5dfc18-7a49-4f4d-a949-8d33d5c7ac09 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:24.783240 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fa5dfc18-7a49-4f4d-a949-8d33d5c7ac09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:24.790366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7943502-18c7-43bd-85ce-19db4b47b46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:24.790990 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7943502-18c7-43bd-85ce-19db4b47b46b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:24.791278 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7943502-18c7-43bd-85ce-19db4b47b46b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:24.791959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7943502-18c7-43bd-85ce-19db4b47b46b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:24.878861 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7943502-18c7-43bd-85ce-19db4b47b46b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:24.880037 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 114/221] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:45:24.880532 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7943502-18c7-43bd-85ce-19db4b47b46b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:24.884328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:24.919684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5dfc18-7a49-4f4d-a949-8d33d5c7ac09 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:24.984631 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:25.016781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 108/222] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:25.022504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7943502-18c7-43bd-85ce-19db4b47b46b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:25.061128 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5dfc18-7a49-4f4d-a949-8d33d5c7ac09 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:25.100906 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73b00a-d585-4f57-be41-853c064bb0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:25.101554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73b00a-d585-4f57-be41-853c064bb0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:25.101842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73b00a-d585-4f57-be41-853c064bb0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:25.102151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73b00a-d585-4f57-be41-853c064bb0b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:25.118222 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73b00a-d585-4f57-be41-853c064bb0b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:25.123519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-3a73b00a-d585-4f57-be41-853c064bb0b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:45:25.123519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:25.123519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:25.123519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:25.123519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:25.123519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:25.123519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:25.123519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:25.123519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:25.123519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:25.123519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:25.123519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:25.123519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:25.123519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:25.123519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:25.123519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:25.124449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:25.124449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:25.124449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:25.124449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:25.124449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:25.124449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:25.124449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:25.124449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:25.124449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:25.124449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:25.124449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:25.124449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:25.124449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:25.124449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:25.124449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:25.124449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:25.125265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:25.125265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:25.125265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:25.125265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:25.125265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:25.125265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:25.125265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:25.125265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:25.125265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:25.125265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:25.125265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:45:25.125265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:25.129580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 115/223] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:45:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:25.129843 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7943502-18c7-43bd-85ce-19db4b47b46b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:25.132195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:25.136390 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73b00a-d585-4f57-be41-853c064bb0b8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:25.166027 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc94b83-ef3a-48ba-89f4-c475adf1f23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:25.166764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc94b83-ef3a-48ba-89f4-c475adf1f23c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:25.167048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc94b83-ef3a-48ba-89f4-c475adf1f23c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:25.167319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc94b83-ef3a-48ba-89f4-c475adf1f23c None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:25.168466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc94b83-ef3a-48ba-89f4-c475adf1f23c None admin] RBAC: Authorizing `identity:get_project(project_id=2164ed8287f243bc9a4a5347d4dc0b1d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:25.170151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc94b83-ef3a-48ba-89f4-c475adf1f23c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:25.172059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 109/224] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:25 2022] GET /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:25.178004 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc94b83-ef3a-48ba-89f4-c475adf1f23c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:25.274552 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73b00a-d585-4f57-be41-853c064bb0b8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:25.279130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:25.293187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc94b83-ef3a-48ba-89f4-c475adf1f23c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:25.315012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ae568-73d1-4ec0-a031-2c8fdf4da6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:25.315770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ae568-73d1-4ec0-a031-2c8fdf4da6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:25.316158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ae568-73d1-4ec0-a031-2c8fdf4da6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:25.316536 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ae568-73d1-4ec0-a031-2c8fdf4da6e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:25.317320 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ae568-73d1-4ec0-a031-2c8fdf4da6e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:25.317785 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ae568-73d1-4ec0-a031-2c8fdf4da6e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:25.319413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ae568-73d1-4ec0-a031-2c8fdf4da6e8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:25.333898 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 116/225] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:25.334526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104950f7-af7d-4312-af18-5e24e9023bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:25.335215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ae568-73d1-4ec0-a031-2c8fdf4da6e8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:25.335485 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104950f7-af7d-4312-af18-5e24e9023bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:25.336025 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104950f7-af7d-4312-af18-5e24e9023bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:25.336442 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104950f7-af7d-4312-af18-5e24e9023bd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:25.337332 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104950f7-af7d-4312-af18-5e24e9023bd3 None admin] RBAC: Adding query filter params (role.id=d96ee0e293244ae6a04e8ce652aea68e, scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d, user.id=5b5c68ce791b4845af27773a417486fd) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:25.337787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104950f7-af7d-4312-af18-5e24e9023bd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:25.340853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104950f7-af7d-4312-af18-5e24e9023bd3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:25.348694 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 110/226] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:45:25 2022] GET /identity/v3/role_assignments?user.id=5b5c68ce791b4845af27773a417486fd&scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d&role.id=d96ee0e293244ae6a04e8ce652aea68e => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:25.349784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104950f7-af7d-4312-af18-5e24e9023bd3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:25.395108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:25.473899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ae568-73d1-4ec0-a031-2c8fdf4da6e8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:25.496479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104950f7-af7d-4312-af18-5e24e9023bd3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:25.554877 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd244c53-c826-4651-9e62-0d7b6273ac07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:25.557125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd244c53-c826-4651-9e62-0d7b6273ac07 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:25.558953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd244c53-c826-4651-9e62-0d7b6273ac07 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:25.559346 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd244c53-c826-4651-9e62-0d7b6273ac07 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:25.578284 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccc268-1d5a-4032-aa31-4303e10a25f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:25.579933 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccc268-1d5a-4032-aa31-4303e10a25f0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:25.582207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccc268-1d5a-4032-aa31-4303e10a25f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:25.582501 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccc268-1d5a-4032-aa31-4303e10a25f0 None admin] PATH_INFO: `/v3/users/d6d10c0a2e0f4e27a799877ea78a13aa` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:25.583179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd244c53-c826-4651-9e62-0d7b6273ac07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:25.583870 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcccc268-1d5a-4032-aa31-4303e10a25f0 None admin] RBAC: Authorizing `identity:get_user(user_id=d6d10c0a2e0f4e27a799877ea78a13aa)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:25.585383 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcccc268-1d5a-4032-aa31-4303e10a25f0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:25.587384 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 111/227] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:25 2022] GET /identity/v3/users/d6d10c0a2e0f4e27a799877ea78a13aa => generated 283 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:25.589005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-fd244c53-c826-4651-9e62-0d7b6273ac07 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:25.589005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:25.589005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:25.589005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:25.589005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:25.589005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:25.589005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:25.589005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:25.589005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:25.589005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:25.589005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:25.589005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:25.589005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:25.589005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:25.589005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:25.589005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:25.589942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:25.589942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:25.589942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:25.589942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:25.589942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:25.589942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:25.589942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:25.589942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:25.589942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:25.589942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:25.589942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:25.589942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:25.589942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:25.589942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:25.589942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:25.589942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:25.590768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:25.590768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:25.590768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:25.590768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:25.590768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:25.590768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:25.590768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:25.590768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:25.590768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:25.590768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:25.590768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:25.590768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:25.594518 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccc268-1d5a-4032-aa31-4303e10a25f0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:25.595240 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 117/228] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:45:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:25.602589 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd244c53-c826-4651-9e62-0d7b6273ac07 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:25.642027 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:25.768838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd244c53-c826-4651-9e62-0d7b6273ac07 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:25.782044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccc268-1d5a-4032-aa31-4303e10a25f0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:25.786718 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:25.844946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ad5f0-fab6-4f5d-a6e6-97ffa87e69a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:25.845567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ad5f0-fab6-4f5d-a6e6-97ffa87e69a0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:25.845879 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ad5f0-fab6-4f5d-a6e6-97ffa87e69a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:25.846156 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ad5f0-fab6-4f5d-a6e6-97ffa87e69a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:25.846846 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ad5f0-fab6-4f5d-a6e6-97ffa87e69a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:25.847158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ad5f0-fab6-4f5d-a6e6-97ffa87e69a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:25.848548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ad5f0-fab6-4f5d-a6e6-97ffa87e69a0 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:25.851279 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee63fc7-eed7-4cb2-a54d-56c22d29d84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:25.851924 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee63fc7-eed7-4cb2-a54d-56c22d29d84d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:25.852292 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee63fc7-eed7-4cb2-a54d-56c22d29d84d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:25.852571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee63fc7-eed7-4cb2-a54d-56c22d29d84d None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:25.854588 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee63fc7-eed7-4cb2-a54d-56c22d29d84d None admin] RBAC: Authorizing `identity:get_project(project_id=2164ed8287f243bc9a4a5347d4dc0b1d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:25.856013 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee63fc7-eed7-4cb2-a54d-56c22d29d84d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:25.857661 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 112/229] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:25 2022] GET /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:25.863919 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee63fc7-eed7-4cb2-a54d-56c22d29d84d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:25.868840 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 118/230] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:45:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:25.872976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ad5f0-fab6-4f5d-a6e6-97ffa87e69a0 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:26.038331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:26.060137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ad5f0-fab6-4f5d-a6e6-97ffa87e69a0 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:26.075669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee63fc7-eed7-4cb2-a54d-56c22d29d84d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:26.114225 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04ead1-58ad-4f2a-9c22-a48d98933eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:26.115234 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04ead1-58ad-4f2a-9c22-a48d98933eb0 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:26.115728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04ead1-58ad-4f2a-9c22-a48d98933eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:26.116311 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04ead1-58ad-4f2a-9c22-a48d98933eb0 None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d/users/d6d10c0a2e0f4e27a799877ea78a13aa/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:26.125883 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04ead1-58ad-4f2a-9c22-a48d98933eb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2164ed8287f243bc9a4a5347d4dc0b1d, user_id=d6d10c0a2e0f4e27a799877ea78a13aa, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:26.125883 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04ead1-58ad-4f2a-9c22-a48d98933eb0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:26.133904 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7a766-a6f5-40ef-a539-59eb590e7bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:26.134599 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7a766-a6f5-40ef-a539-59eb590e7bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:26.134884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7a766-a6f5-40ef-a539-59eb590e7bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:26.135157 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7a766-a6f5-40ef-a539-59eb590e7bb8 None admin] PATH_INFO: `/v3/users/d6d10c0a2e0f4e27a799877ea78a13aa` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:26.136418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7a766-a6f5-40ef-a539-59eb590e7bb8 None admin] RBAC: Authorizing `identity:get_user(user_id=d6d10c0a2e0f4e27a799877ea78a13aa)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:26.137943 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7a766-a6f5-40ef-a539-59eb590e7bb8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:26.139915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 119/231] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:25 2022] GET /identity/v3/users/d6d10c0a2e0f4e27a799877ea78a13aa => generated 283 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:26.145603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7a766-a6f5-40ef-a539-59eb590e7bb8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:26.204497 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 113/232] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:45:25 2022] PUT /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d/users/d6d10c0a2e0f4e27a799877ea78a13aa/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:45:26.272363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7a766-a6f5-40ef-a539-59eb590e7bb8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:26.311066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6360852-2de1-401e-8e18-c68b1b29d9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:26.311702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6360852-2de1-401e-8e18-c68b1b29d9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:26.311988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6360852-2de1-401e-8e18-c68b1b29d9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:26.312264 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6360852-2de1-401e-8e18-c68b1b29d9c5 None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:26.313481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6360852-2de1-401e-8e18-c68b1b29d9c5 None admin] RBAC: Authorizing `identity:get_project(project_id=2164ed8287f243bc9a4a5347d4dc0b1d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:26.315062 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6360852-2de1-401e-8e18-c68b1b29d9c5 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:26.317143 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 120/233] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:26 2022] GET /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:26.323192 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04ead1-58ad-4f2a-9c22-a48d98933eb0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:26.377453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:26.472358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04ead1-58ad-4f2a-9c22-a48d98933eb0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:26.510315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a853b76-80c5-41ba-a671-d9bcd4e174a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:26.511335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a853b76-80c5-41ba-a671-d9bcd4e174a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:26.511815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a853b76-80c5-41ba-a671-d9bcd4e174a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:26.512316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a853b76-80c5-41ba-a671-d9bcd4e174a1 None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d/users/d6d10c0a2e0f4e27a799877ea78a13aa/roles/1ab8c505dc344769abc347f66e529527` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:26.514823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a853b76-80c5-41ba-a671-d9bcd4e174a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=2164ed8287f243bc9a4a5347d4dc0b1d, user_id=d6d10c0a2e0f4e27a799877ea78a13aa, role_id=1ab8c505dc344769abc347f66e529527)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:26.516539 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a853b76-80c5-41ba-a671-d9bcd4e174a1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:26.598624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 114/234] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:45:26 2022] PUT /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d/users/d6d10c0a2e0f4e27a799877ea78a13aa/roles/1ab8c505dc344769abc347f66e529527 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:45:27.318002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ab7b0-918d-40a6-a93e-44cffeccd4f1 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:27.318520 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ab7b0-918d-40a6-a93e-44cffeccd4f1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:27.318596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ab7b0-918d-40a6-a93e-44cffeccd4f1 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:27.319907 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 121/235] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:27.325370 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e7e44-31f0-408f-b8e6-279f0730d1ca None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:27.326237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e7e44-31f0-408f-b8e6-279f0730d1ca None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:27.326716 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e7e44-31f0-408f-b8e6-279f0730d1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:27.359003 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6f0e7e44-31f0-408f-b8e6-279f0730d1ca None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:27.360242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6f0e7e44-31f0-408f-b8e6-279f0730d1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:27.442577 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 115/236] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:45:27.607616 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df38f4b-e322-4ab3-84ae-53c0cc14c511 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:27.607950 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df38f4b-e322-4ab3-84ae-53c0cc14c511 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:27.608218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df38f4b-e322-4ab3-84ae-53c0cc14c511 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:27.609562 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 122/237] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:27.614824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af44842-4cf1-4478-aa9c-2d2999bcb114 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:27.615122 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af44842-4cf1-4478-aa9c-2d2999bcb114 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:27.615406 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af44842-4cf1-4478-aa9c-2d2999bcb114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:27.638275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1af44842-4cf1-4478-aa9c-2d2999bcb114 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:27.640008 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1af44842-4cf1-4478-aa9c-2d2999bcb114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:27.662489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:27.736170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 116/238] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:27.830569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df38f4b-e322-4ab3-84ae-53c0cc14c511 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:27.869480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:27.883562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da113245-d53c-4f93-b3ac-c6ac832c82fa None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:27.883852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da113245-d53c-4f93-b3ac-c6ac832c82fa None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:27.887614 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da113245-d53c-4f93-b3ac-c6ac832c82fa None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:27.887614 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 117/239] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:45:27.894491 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cd3cb-17e4-4479-a997-ed91e4468e69 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:27.894769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cd3cb-17e4-4479-a997-ed91e4468e69 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:27.895052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cd3cb-17e4-4479-a997-ed91e4468e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:27.929881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-608cd3cb-17e4-4479-a997-ed91e4468e69 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:27.931133 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-608cd3cb-17e4-4479-a997-ed91e4468e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:28.011957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df38f4b-e322-4ab3-84ae-53c0cc14c511 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:28.050802 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 118/240] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:45:28.060656 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d095a-34ff-4e4b-81eb-1292eb72679d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:28.061821 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d095a-34ff-4e4b-81eb-1292eb72679d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:28.062147 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d095a-34ff-4e4b-81eb-1292eb72679d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:28.062427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d095a-34ff-4e4b-81eb-1292eb72679d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:28.072983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d095a-34ff-4e4b-81eb-1292eb72679d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:28.078673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-ba9d095a-34ff-4e4b-81eb-1292eb72679d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 11:45:28.078673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:28.078673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:28.078673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:28.078673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:28.078673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:28.078673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:28.078673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:28.078673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:28.078673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:28.078673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:28.078673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:28.078673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:28.078673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:28.078673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:28.078673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:28.079548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:28.079548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:28.079548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:28.079548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:28.079548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:28.079548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:28.079548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:28.079548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:28.079548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:28.079548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:28.079548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:28.079548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:28.079548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:28.079548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:28.079548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:28.079548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:28.080880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:28.080880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:28.080880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:28.080880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:28.080880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:28.080880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:28.080880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:28.080880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:28.080880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:28.080880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:28.080880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 11:45:28.080880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:28.080880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 123/241] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:28.085800 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cd3cb-17e4-4479-a997-ed91e4468e69 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:28.096781 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d095a-34ff-4e4b-81eb-1292eb72679d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:28.194373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:28.214394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cd3cb-17e4-4479-a997-ed91e4468e69 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:28.252403 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ca8c8-fcaf-43f9-9f04-5203ee22e04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:28.253024 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ca8c8-fcaf-43f9-9f04-5203ee22e04d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:28.253310 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ca8c8-fcaf-43f9-9f04-5203ee22e04d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:28.253592 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ca8c8-fcaf-43f9-9f04-5203ee22e04d None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:28.254289 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ca8c8-fcaf-43f9-9f04-5203ee22e04d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:28.254599 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ca8c8-fcaf-43f9-9f04-5203ee22e04d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:28.256154 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ca8c8-fcaf-43f9-9f04-5203ee22e04d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:28.266967 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 119/242] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:28.272907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ca8c8-fcaf-43f9-9f04-5203ee22e04d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:28.303846 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:28.350888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d095a-34ff-4e4b-81eb-1292eb72679d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:28.384059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ca8c8-fcaf-43f9-9f04-5203ee22e04d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:28.420002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb554a-5b32-4da5-8540-e3ecbcac0ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:28.420681 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb554a-5b32-4da5-8540-e3ecbcac0ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:28.420968 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb554a-5b32-4da5-8540-e3ecbcac0ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:28.421250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb554a-5b32-4da5-8540-e3ecbcac0ec8 None admin] PATH_INFO: `/v3/users/c01d7e03f5e54796bc601b39c20887df` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:28.432102 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66946ff9-0eab-4a16-85eb-ea3d51d2aa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:28.433447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66946ff9-0eab-4a16-85eb-ea3d51d2aa78 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:28.433920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66946ff9-0eab-4a16-85eb-ea3d51d2aa78 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:28.434608 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66946ff9-0eab-4a16-85eb-ea3d51d2aa78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:28.435709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb554a-5b32-4da5-8540-e3ecbcac0ec8 None admin] RBAC: Authorizing `identity:get_user(user_id=c01d7e03f5e54796bc601b39c20887df)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:28.436281 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66946ff9-0eab-4a16-85eb-ea3d51d2aa78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:28.437425 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb554a-5b32-4da5-8540-e3ecbcac0ec8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:28.438082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66946ff9-0eab-4a16-85eb-ea3d51d2aa78 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:28.439624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 120/243] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:28 2022] GET /identity/v3/users/c01d7e03f5e54796bc601b39c20887df => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:28.440210 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 124/244] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:45:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:28.441264 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb554a-5b32-4da5-8540-e3ecbcac0ec8 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:28.447915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:28.450779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66946ff9-0eab-4a16-85eb-ea3d51d2aa78 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:28.560105 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb554a-5b32-4da5-8540-e3ecbcac0ec8 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:28.578776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66946ff9-0eab-4a16-85eb-ea3d51d2aa78 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:28.596505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a474ea-5f6b-4eac-8b60-c0f4eb3e4e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:28.597143 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a474ea-5f6b-4eac-8b60-c0f4eb3e4e4d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:28.597425 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a474ea-5f6b-4eac-8b60-c0f4eb3e4e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:28.597698 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a474ea-5f6b-4eac-8b60-c0f4eb3e4e4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:28.608448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a474ea-5f6b-4eac-8b60-c0f4eb3e4e4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:28.613312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-c7a474ea-5f6b-4eac-8b60-c0f4eb3e4e4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:28.613312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:28.613312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:28.613312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:28.613312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:28.613312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:28.613312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:28.613312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:28.613312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:28.613312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:28.613312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:28.613312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:28.613312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:28.613312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:28.613312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:28.613312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:28.614214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:28.614214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:28.614214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:28.614214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:28.614214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:28.614214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:28.614214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:28.614214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:28.614214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:28.614214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:28.614214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:28.614214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:28.614214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:28.614214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:28.614214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:28.614214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:28.615033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:28.615033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:28.615033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:28.615033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:28.615033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:28.615033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:28.615033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:28.615033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:28.615033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:28.615033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:28.615033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:28.615033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:28.615033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 121/245] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:28.615684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a474ea-5f6b-4eac-8b60-c0f4eb3e4e4d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:28.624543 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82257e38-5495-4ce4-92f4-44895e4f9367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:28.625689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82257e38-5495-4ce4-92f4-44895e4f9367 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:28.626026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82257e38-5495-4ce4-92f4-44895e4f9367 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:28.626310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82257e38-5495-4ce4-92f4-44895e4f9367 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:28.627448 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82257e38-5495-4ce4-92f4-44895e4f9367 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:28.629438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82257e38-5495-4ce4-92f4-44895e4f9367 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:28.680608 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:28.714382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 125/246] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 14 11:45:28 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:45:28.714382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82257e38-5495-4ce4-92f4-44895e4f9367 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:28.721677 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a474ea-5f6b-4eac-8b60-c0f4eb3e4e4d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:28.727646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:28.764275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f182f-809f-4c5b-85f9-eb64fedbeed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:28.765677 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f182f-809f-4c5b-85f9-eb64fedbeed4 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:28.766094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f182f-809f-4c5b-85f9-eb64fedbeed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:28.766470 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f182f-809f-4c5b-85f9-eb64fedbeed4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:28.876837 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f182f-809f-4c5b-85f9-eb64fedbeed4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:28.879503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f182f-809f-4c5b-85f9-eb64fedbeed4 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:28.892661 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82257e38-5495-4ce4-92f4-44895e4f9367 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:28.940986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d9aad-2766-4b3a-8f4a-ba42a4f40be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:28.942208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d9aad-2766-4b3a-8f4a-ba42a4f40be1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:28.942703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d9aad-2766-4b3a-8f4a-ba42a4f40be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:28.943520 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d9aad-2766-4b3a-8f4a-ba42a4f40be1 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:28.944421 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d9aad-2766-4b3a-8f4a-ba42a4f40be1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:28.944954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d9aad-2766-4b3a-8f4a-ba42a4f40be1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:28.947184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d9aad-2766-4b3a-8f4a-ba42a4f40be1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:28.955774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:28.962180 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 126/247] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:28.968065 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27695e21-64b4-4450-8711-b33924a4b90b None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:28.968065 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27695e21-64b4-4450-8711-b33924a4b90b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:28.968065 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27695e21-64b4-4450-8711-b33924a4b90b None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:28.968065 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 127/248] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:28.970553 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27695e21-64b4-4450-8711-b33924a4b90b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:29.007227 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 122/249] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:29.008627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98899cdc-5e18-4428-8b17-223ea93b702a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:29.009133 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98899cdc-5e18-4428-8b17-223ea93b702a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:29.009238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98899cdc-5e18-4428-8b17-223ea93b702a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:29.029287 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-98899cdc-5e18-4428-8b17-223ea93b702a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:29.030531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-98899cdc-5e18-4428-8b17-223ea93b702a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:29.067523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:29.115309 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27695e21-64b4-4450-8711-b33924a4b90b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:29.117047 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 123/250] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:29.118165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98899cdc-5e18-4428-8b17-223ea93b702a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:29.159687 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875326fe-b188-4ffb-ab0b-1ded6b52ecce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:29.160856 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875326fe-b188-4ffb-ab0b-1ded6b52ecce None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:29.161301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875326fe-b188-4ffb-ab0b-1ded6b52ecce None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:29.161725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875326fe-b188-4ffb-ab0b-1ded6b52ecce None admin] PATH_INFO: `/v3/users/5b5c68ce791b4845af27773a417486fd` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:29.163352 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875326fe-b188-4ffb-ab0b-1ded6b52ecce None admin] RBAC: Authorizing `identity:get_user(user_id=5b5c68ce791b4845af27773a417486fd)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:29.166552 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875326fe-b188-4ffb-ab0b-1ded6b52ecce None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:29.168375 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 128/251] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:28 2022] GET /identity/v3/users/5b5c68ce791b4845af27773a417486fd => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:29.176122 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875326fe-b188-4ffb-ab0b-1ded6b52ecce None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:29.245291 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:29.261167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98899cdc-5e18-4428-8b17-223ea93b702a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:29.317929 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fb6a1-39fb-4d1d-b010-82dee119c547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:29.318515 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875326fe-b188-4ffb-ab0b-1ded6b52ecce None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:29.318622 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fb6a1-39fb-4d1d-b010-82dee119c547 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:29.318853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fb6a1-39fb-4d1d-b010-82dee119c547 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:29.319132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fb6a1-39fb-4d1d-b010-82dee119c547 None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:29.320280 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fb6a1-39fb-4d1d-b010-82dee119c547 None admin] RBAC: Authorizing `identity:get_project(project_id=2164ed8287f243bc9a4a5347d4dc0b1d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:29.323319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fb6a1-39fb-4d1d-b010-82dee119c547 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:29.329599 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 124/252] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:29 2022] GET /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:29.339328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fb6a1-39fb-4d1d-b010-82dee119c547 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:29.367268 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79811697-96d5-4bca-b988-d7d27430a719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:29.368003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79811697-96d5-4bca-b988-d7d27430a719 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:29.368290 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79811697-96d5-4bca-b988-d7d27430a719 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:29.368567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79811697-96d5-4bca-b988-d7d27430a719 None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:29.370788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79811697-96d5-4bca-b988-d7d27430a719 None admin] RBAC: Authorizing `identity:get_project(project_id=2164ed8287f243bc9a4a5347d4dc0b1d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:29.371716 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79811697-96d5-4bca-b988-d7d27430a719 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:29.373740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 129/253] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:29 2022] GET /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:29.379637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79811697-96d5-4bca-b988-d7d27430a719 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:29.397008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:29.452984 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fb6a1-39fb-4d1d-b010-82dee119c547 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:29.488661 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79811697-96d5-4bca-b988-d7d27430a719 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:29.496331 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb963a5-b783-4e9d-9038-43200631790e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:29.497425 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb963a5-b783-4e9d-9038-43200631790e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:29.498052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb963a5-b783-4e9d-9038-43200631790e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:29.498610 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb963a5-b783-4e9d-9038-43200631790e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:29.499662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb963a5-b783-4e9d-9038-43200631790e None admin] RBAC: Adding query filter params (role.id=f792865844594321a8a67b38335cdccb, scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d, user.id=c01d7e03f5e54796bc601b39c20887df) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:29.500313 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb963a5-b783-4e9d-9038-43200631790e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:29.502092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb963a5-b783-4e9d-9038-43200631790e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:29.508742 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 125/254] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:45:29 2022] GET /identity/v3/role_assignments?user.id=c01d7e03f5e54796bc601b39c20887df&scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d&role.id=f792865844594321a8a67b38335cdccb => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:29.509968 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb963a5-b783-4e9d-9038-43200631790e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:29.539810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:29.543792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d0635-570d-45a7-98e7-a40e76b9b1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:29.544493 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d0635-570d-45a7-98e7-a40e76b9b1b1 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:29.545006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d0635-570d-45a7-98e7-a40e76b9b1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:29.545110 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d0635-570d-45a7-98e7-a40e76b9b1b1 None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d/users/5b5c68ce791b4845af27773a417486fd/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:29.551677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87d0635-570d-45a7-98e7-a40e76b9b1b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=2164ed8287f243bc9a4a5347d4dc0b1d, user_id=5b5c68ce791b4845af27773a417486fd, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:29.553319 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87d0635-570d-45a7-98e7-a40e76b9b1b1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:29.629572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb963a5-b783-4e9d-9038-43200631790e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:29.669620 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254cd91f-aba5-47e8-bce8-4a79ef23364b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:29.671271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cd91f-aba5-47e8-bce8-4a79ef23364b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:29.671752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cd91f-aba5-47e8-bce8-4a79ef23364b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:29.672293 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cd91f-aba5-47e8-bce8-4a79ef23364b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:29.673147 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 130/255] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:45:29 2022] PUT /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d/users/5b5c68ce791b4845af27773a417486fd/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 11:45:29.680874 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35658447-2598-47fe-b07c-60e48d90b866 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:29.681187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35658447-2598-47fe-b07c-60e48d90b866 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:29.681454 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35658447-2598-47fe-b07c-60e48d90b866 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:29.683131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 131/256] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:29.687486 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254cd91f-aba5-47e8-bce8-4a79ef23364b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:29.689475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e74b3-e73e-49e7-82bc-6404928d279a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:29.689773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e74b3-e73e-49e7-82bc-6404928d279a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:29.690138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e74b3-e73e-49e7-82bc-6404928d279a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:29.693787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-254cd91f-aba5-47e8-bce8-4a79ef23364b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:45:29.693787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:29.693787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:29.693787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:29.693787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:29.693787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:29.693787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:29.693787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:29.693787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:29.693787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:29.693787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:29.693787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:29.693787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:29.693787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:29.693787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:29.693787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:29.699241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:29.699241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:29.699241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:29.699241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:29.699241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:29.699241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:29.699241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:29.699241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:29.699241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:29.699241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:29.699241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:29.699241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:29.699241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:29.699241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:29.699241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:29.699241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:29.700249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:29.700249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:29.700249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:29.700249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:29.700249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:29.700249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:29.700249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:29.700249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:29.700249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:29.700249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:29.700249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:45:29.700249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:29.700249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 126/257] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:45:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:29.704684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254cd91f-aba5-47e8-bce8-4a79ef23364b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:29.735217 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c31e74b3-e73e-49e7-82bc-6404928d279a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:29.736535 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c31e74b3-e73e-49e7-82bc-6404928d279a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:29.841906 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:29.854261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:29.863295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 132/258] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:45:29.867804 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254cd91f-aba5-47e8-bce8-4a79ef23364b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:29.909616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6fd6c-b4eb-4db7-acf4-2f0fb1f94f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:29.910895 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6fd6c-b4eb-4db7-acf4-2f0fb1f94f1a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:29.911388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6fd6c-b4eb-4db7-acf4-2f0fb1f94f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:29.911825 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6fd6c-b4eb-4db7-acf4-2f0fb1f94f1a None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:29.912671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6fd6c-b4eb-4db7-acf4-2f0fb1f94f1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:29.913284 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6fd6c-b4eb-4db7-acf4-2f0fb1f94f1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:29.914972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6fd6c-b4eb-4db7-acf4-2f0fb1f94f1a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:29.930003 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 127/259] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:29.932984 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e74b3-e73e-49e7-82bc-6404928d279a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:30.056607 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e74b3-e73e-49e7-82bc-6404928d279a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:30.094313 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c29ac8-0489-4b73-aced-cfe2ce78913f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:30.094966 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c29ac8-0489-4b73-aced-cfe2ce78913f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:30.095254 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c29ac8-0489-4b73-aced-cfe2ce78913f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:30.095537 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c29ac8-0489-4b73-aced-cfe2ce78913f None admin] PATH_INFO: `/v3/users/d6d10c0a2e0f4e27a799877ea78a13aa` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:30.096704 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c29ac8-0489-4b73-aced-cfe2ce78913f None admin] RBAC: Authorizing `identity:get_user(user_id=d6d10c0a2e0f4e27a799877ea78a13aa)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:30.098223 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c29ac8-0489-4b73-aced-cfe2ce78913f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:30.100133 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 133/260] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:29 2022] GET /identity/v3/users/d6d10c0a2e0f4e27a799877ea78a13aa => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:30.105762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6fd6c-b4eb-4db7-acf4-2f0fb1f94f1a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:30.271349 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6fd6c-b4eb-4db7-acf4-2f0fb1f94f1a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:30.318499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58a1bd-28b8-4a67-aef5-ef5b95b89ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:30.319111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58a1bd-28b8-4a67-aef5-ef5b95b89ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:30.319390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58a1bd-28b8-4a67-aef5-ef5b95b89ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:30.319665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58a1bd-28b8-4a67-aef5-ef5b95b89ed8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:30.335807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:30.391786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58a1bd-28b8-4a67-aef5-ef5b95b89ed8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:30.393177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58a1bd-28b8-4a67-aef5-ef5b95b89ed8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:30.428702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c29ac8-0489-4b73-aced-cfe2ce78913f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:30.533877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 128/261] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:30.539799 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58a1bd-28b8-4a67-aef5-ef5b95b89ed8 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:30.562247 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:30.616812 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:30.623462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c29ac8-0489-4b73-aced-cfe2ce78913f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:30.687483 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396df4b-dd34-4baa-a561-c0742a086c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:30.688695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396df4b-dd34-4baa-a561-c0742a086c75 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:30.690116 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396df4b-dd34-4baa-a561-c0742a086c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:30.691751 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396df4b-dd34-4baa-a561-c0742a086c75 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:30.711783 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58a1bd-28b8-4a67-aef5-ef5b95b89ed8 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:30.713667 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396df4b-dd34-4baa-a561-c0742a086c75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:30.724860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2396df4b-dd34-4baa-a561-c0742a086c75 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:30.724860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:30.724860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:30.724860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:30.724860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:30.724860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:30.724860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:30.724860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:30.724860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:30.724860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:30.724860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:30.724860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:30.724860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:30.724860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:30.724860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:30.724860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:30.725758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:30.725758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:30.725758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:30.725758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:30.725758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:30.725758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:30.725758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:30.725758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:30.725758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:30.725758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:30.725758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:30.725758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:30.725758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:30.725758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:30.725758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:30.725758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:30.726591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:30.726591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:30.726591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:30.726591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:30.726591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:30.726591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:30.726591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:30.726591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:30.726591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:30.726591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:30.726591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:30.726591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:30.729097 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 134/262] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:45:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:45:30.734329 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396df4b-dd34-4baa-a561-c0742a086c75 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:30.781734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc502aba-33e5-4fd3-a622-a134dbdb425f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:30.782408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc502aba-33e5-4fd3-a622-a134dbdb425f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:30.782693 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc502aba-33e5-4fd3-a622-a134dbdb425f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:30.782970 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc502aba-33e5-4fd3-a622-a134dbdb425f None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:30.786653 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc502aba-33e5-4fd3-a622-a134dbdb425f None admin] RBAC: Authorizing `identity:get_project(project_id=2164ed8287f243bc9a4a5347d4dc0b1d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:30.788101 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc502aba-33e5-4fd3-a622-a134dbdb425f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:30.790329 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 129/263] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:30 2022] GET /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:30.796003 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc502aba-33e5-4fd3-a622-a134dbdb425f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:30.867272 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396df4b-dd34-4baa-a561-c0742a086c75 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:30.908837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:30.926199 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffe6ab-1e17-4f9f-afce-9e962c80726e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:30.927202 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffe6ab-1e17-4f9f-afce-9e962c80726e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:30.927491 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffe6ab-1e17-4f9f-afce-9e962c80726e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:30.928211 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffe6ab-1e17-4f9f-afce-9e962c80726e None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:30.928430 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffe6ab-1e17-4f9f-afce-9e962c80726e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:30.928731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffe6ab-1e17-4f9f-afce-9e962c80726e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:30.930820 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffe6ab-1e17-4f9f-afce-9e962c80726e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:30.946608 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 135/264] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:45:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:30.954615 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffe6ab-1e17-4f9f-afce-9e962c80726e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:31.075506 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc502aba-33e5-4fd3-a622-a134dbdb425f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:31.098077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:31.105262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffe6ab-1e17-4f9f-afce-9e962c80726e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:31.143351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434715d-442a-4ff6-ae3b-3ba3d65fce5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:31.143993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434715d-442a-4ff6-ae3b-3ba3d65fce5a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:31.144274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434715d-442a-4ff6-ae3b-3ba3d65fce5a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:31.144553 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434715d-442a-4ff6-ae3b-3ba3d65fce5a None admin] PATH_INFO: `/v3/users/d6d10c0a2e0f4e27a799877ea78a13aa` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:31.145731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434715d-442a-4ff6-ae3b-3ba3d65fce5a None admin] RBAC: Authorizing `identity:get_user(user_id=d6d10c0a2e0f4e27a799877ea78a13aa)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:31.147457 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434715d-442a-4ff6-ae3b-3ba3d65fce5a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:31.149036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 136/265] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:30 2022] GET /identity/v3/users/d6d10c0a2e0f4e27a799877ea78a13aa => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:31.154434 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434715d-442a-4ff6-ae3b-3ba3d65fce5a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:31.154582 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca42c6-fc86-41c8-af54-93f17558f34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:31.155226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca42c6-fc86-41c8-af54-93f17558f34b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:31.155516 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca42c6-fc86-41c8-af54-93f17558f34b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:31.155800 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca42c6-fc86-41c8-af54-93f17558f34b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:31.156545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca42c6-fc86-41c8-af54-93f17558f34b None admin] RBAC: Adding query filter params (role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d, user.id=d6d10c0a2e0f4e27a799877ea78a13aa) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:31.156860 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca42c6-fc86-41c8-af54-93f17558f34b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:31.162426 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca42c6-fc86-41c8-af54-93f17558f34b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:31.180893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 130/266] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:45:30 2022] GET /identity/v3/role_assignments?user.id=d6d10c0a2e0f4e27a799877ea78a13aa&scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 608 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:31.240737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:31.313370 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434715d-442a-4ff6-ae3b-3ba3d65fce5a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:31.372592 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb7cdc-e2d5-43a1-846b-553cfb5bde3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:31.373232 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb7cdc-e2d5-43a1-846b-553cfb5bde3d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:31.373517 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb7cdc-e2d5-43a1-846b-553cfb5bde3d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:31.374371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb7cdc-e2d5-43a1-846b-553cfb5bde3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:31.489632 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb7cdc-e2d5-43a1-846b-553cfb5bde3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:31.491211 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb7cdc-e2d5-43a1-846b-553cfb5bde3d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:31.568315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:31.625741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 137/267] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:31.628091 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca42c6-fc86-41c8-af54-93f17558f34b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:31.739872 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca42c6-fc86-41c8-af54-93f17558f34b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:31.774621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1271ae-91c2-410d-b50d-f6d094fe9872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:31.775176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1271ae-91c2-410d-b50d-f6d094fe9872 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:31.775454 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1271ae-91c2-410d-b50d-f6d094fe9872 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:31.775729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1271ae-91c2-410d-b50d-f6d094fe9872 None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:31.776848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1271ae-91c2-410d-b50d-f6d094fe9872 None admin] RBAC: Authorizing `identity:get_project(project_id=2164ed8287f243bc9a4a5347d4dc0b1d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:31.778313 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1271ae-91c2-410d-b50d-f6d094fe9872 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:31.780781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 131/268] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:31 2022] GET /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:31.786661 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb7cdc-e2d5-43a1-846b-553cfb5bde3d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:31.893249 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb7cdc-e2d5-43a1-846b-553cfb5bde3d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:31.928657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a0bf4-af60-466e-8657-fb950bf5134c None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:31.936296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a0bf4-af60-466e-8657-fb950bf5134c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:31.936296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a0bf4-af60-466e-8657-fb950bf5134c None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:31.936296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 132/269] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:31.936296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ff074-bac8-46c0-b280-d7cb419c6b47 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:31.937053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ff074-bac8-46c0-b280-d7cb419c6b47 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:31.937540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ff074-bac8-46c0-b280-d7cb419c6b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:31.952775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a86c3f-dd65-4bfb-b7b5-c9656e5e4ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:31.953402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a86c3f-dd65-4bfb-b7b5-c9656e5e4ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:31.953687 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a86c3f-dd65-4bfb-b7b5-c9656e5e4ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:31.953989 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a86c3f-dd65-4bfb-b7b5-c9656e5e4ad9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:31.956274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a86c3f-dd65-4bfb-b7b5-c9656e5e4ad9 None admin] RBAC: Adding query filter params (role.id=1ab8c505dc344769abc347f66e529527, scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d, user.id=d6d10c0a2e0f4e27a799877ea78a13aa) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:31.956274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a86c3f-dd65-4bfb-b7b5-c9656e5e4ad9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:31.956523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a86c3f-dd65-4bfb-b7b5-c9656e5e4ad9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:31.966405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 138/270] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:45:31 2022] GET /identity/v3/role_assignments?user.id=d6d10c0a2e0f4e27a799877ea78a13aa&scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d&role.id=1ab8c505dc344769abc347f66e529527 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:31.967320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f53ff074-bac8-46c0-b280-d7cb419c6b47 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:31.978051 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f53ff074-bac8-46c0-b280-d7cb419c6b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:32.036632 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:32.076528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 133/271] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:45:32.390054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d241d0-5a11-487f-be4b-9c921c0322d2 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:32.390425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d241d0-5a11-487f-be4b-9c921c0322d2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:32.390627 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d241d0-5a11-487f-be4b-9c921c0322d2 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:32.392406 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 139/272] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:32.398002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb56d0-bee2-4dc5-a1ef-f7cbf4071908 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:32.398299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb56d0-bee2-4dc5-a1ef-f7cbf4071908 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:32.398584 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb56d0-bee2-4dc5-a1ef-f7cbf4071908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:32.410975 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d241d0-5a11-487f-be4b-9c921c0322d2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:32.430647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-39eb56d0-bee2-4dc5-a1ef-f7cbf4071908 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:32.432156 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-39eb56d0-bee2-4dc5-a1ef-f7cbf4071908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:32.433977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:32.524287 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 134/273] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:32.525615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1bc21-07a5-488f-b730-400df7cc9345 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:32.525951 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1bc21-07a5-488f-b730-400df7cc9345 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:32.526236 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1bc21-07a5-488f-b730-400df7cc9345 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:32.527479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 135/274] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:32.532691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212ab88-d637-4b1d-9fe1-e95c4d228b72 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:32.533107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212ab88-d637-4b1d-9fe1-e95c4d228b72 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:32.533487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212ab88-d637-4b1d-9fe1-e95c4d228b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:32.549658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d241d0-5a11-487f-be4b-9c921c0322d2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:32.554756 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d212ab88-d637-4b1d-9fe1-e95c4d228b72 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:32.556111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d212ab88-d637-4b1d-9fe1-e95c4d228b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:32.600283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8b357-3f6e-4849-83e6-d060504623b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:32.601193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8b357-3f6e-4849-83e6-d060504623b2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:32.601616 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8b357-3f6e-4849-83e6-d060504623b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:32.602139 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8b357-3f6e-4849-83e6-d060504623b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:32.616924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8b357-3f6e-4849-83e6-d060504623b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:32.625596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-24b8b357-3f6e-4849-83e6-d060504623b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:32.625596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:32.625596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:32.625596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:32.625596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:32.625596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:32.625596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:32.625596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:32.625596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:32.625596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:32.625596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:32.625596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:32.625596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:32.625596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:32.625596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:32.625596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:32.626726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:32.626726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:32.626726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:32.626726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:32.626726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:32.626726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:32.626726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:32.626726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:32.626726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:32.626726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:32.626726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:32.626726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:32.626726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:32.626726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:32.626726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:32.626726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:32.627715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:32.627715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:32.627715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:32.627715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:32.627715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:32.627715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:32.627715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:32.627715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:32.627715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:32.627715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:32.627715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:32.627715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:32.630128 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 140/275] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:45:32.635831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8b357-3f6e-4849-83e6-d060504623b2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:32.638974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 136/276] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:45:32.718592 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:32.740104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8b357-3f6e-4849-83e6-d060504623b2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:32.775247 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b2088-ef1f-4a38-8a31-c770ba24737b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:32.776111 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b2088-ef1f-4a38-8a31-c770ba24737b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:32.776588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b2088-ef1f-4a38-8a31-c770ba24737b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:32.777000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b2088-ef1f-4a38-8a31-c770ba24737b None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:32.777798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b2088-ef1f-4a38-8a31-c770ba24737b None admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:32.778276 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b2088-ef1f-4a38-8a31-c770ba24737b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:32.780411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b2088-ef1f-4a38-8a31-c770ba24737b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:32.789239 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 141/277] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:32.795129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212ab88-d637-4b1d-9fe1-e95c4d228b72 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:32.808328 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b2088-ef1f-4a38-8a31-c770ba24737b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:32.845152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:32.918479 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b2088-ef1f-4a38-8a31-c770ba24737b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:32.941285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212ab88-d637-4b1d-9fe1-e95c4d228b72 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:32.958330 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e786545-79b9-4eee-b372-fd556fc033e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:32.959198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e786545-79b9-4eee-b372-fd556fc033e3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:32.959487 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e786545-79b9-4eee-b372-fd556fc033e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:32.959761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e786545-79b9-4eee-b372-fd556fc033e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:32.961231 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e786545-79b9-4eee-b372-fd556fc033e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:32.963259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e786545-79b9-4eee-b372-fd556fc033e3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:32.970765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 142/278] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:45:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:32.970765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e786545-79b9-4eee-b372-fd556fc033e3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:32.977602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee812ce3-324f-4c57-b459-9173b992ac9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:32.978302 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee812ce3-324f-4c57-b459-9173b992ac9a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:32.978605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee812ce3-324f-4c57-b459-9173b992ac9a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:32.978858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee812ce3-324f-4c57-b459-9173b992ac9a None admin] PATH_INFO: `/v3/users/6972e51a7df34f1cac25e6461f876ac7` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:32.985891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:32.993427 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee812ce3-324f-4c57-b459-9173b992ac9a None admin] RBAC: Authorizing `identity:get_user(user_id=6972e51a7df34f1cac25e6461f876ac7)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:33.000430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee812ce3-324f-4c57-b459-9173b992ac9a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:33.000430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 137/279] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:32 2022] GET /identity/v3/users/6972e51a7df34f1cac25e6461f876ac7 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:33.000430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee812ce3-324f-4c57-b459-9173b992ac9a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:33.081232 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e786545-79b9-4eee-b372-fd556fc033e3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:33.096556 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:33.103081 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee812ce3-324f-4c57-b459-9173b992ac9a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:33.113778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d9f41-66d9-4c4d-a07d-b58007114b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:33.114434 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d9f41-66d9-4c4d-a07d-b58007114b76 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:33.114720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d9f41-66d9-4c4d-a07d-b58007114b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:33.115000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d9f41-66d9-4c4d-a07d-b58007114b76 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:33.125654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d9f41-66d9-4c4d-a07d-b58007114b76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:33.130937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-351d9f41-66d9-4c4d-a07d-b58007114b76 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 11:45:33.130937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:33.130937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:33.130937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:33.130937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:33.130937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:33.130937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:33.130937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:33.130937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:33.130937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:33.130937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:33.130937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:33.130937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:33.130937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:33.130937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:33.130937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:33.131880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:33.131880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:33.131880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:33.131880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:33.131880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:33.131880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:33.131880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:33.131880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:33.131880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:33.131880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:33.131880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:33.131880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:33.131880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:33.131880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:33.131880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:33.131880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:33.132669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:33.132669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:33.132669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:33.132669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:33.132669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:33.132669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:33.132669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:33.132669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:33.132669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:33.132669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:33.132669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 11:45:33.132669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:33.132669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 143/280] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:33.133330 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d9f41-66d9-4c4d-a07d-b58007114b76 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:33.139819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ac744-2d56-41df-a42a-2b7b2e7bac0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:33.140421 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ac744-2d56-41df-a42a-2b7b2e7bac0b None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:33.140699 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ac744-2d56-41df-a42a-2b7b2e7bac0b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:33.140969 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ac744-2d56-41df-a42a-2b7b2e7bac0b None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:33.141619 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ac744-2d56-41df-a42a-2b7b2e7bac0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:33.143495 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ac744-2d56-41df-a42a-2b7b2e7bac0b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:33.196954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:33.219809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 138/281] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 14 11:45:32 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:45:33.221201 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdbee3-b063-4834-9420-1b92c83fc130 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:33.221498 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdbee3-b063-4834-9420-1b92c83fc130 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:33.221786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdbee3-b063-4834-9420-1b92c83fc130 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:33.223070 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 139/282] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:33.227747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdbee3-b063-4834-9420-1b92c83fc130 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:33.242094 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d9f41-66d9-4c4d-a07d-b58007114b76 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:33.286609 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb61f6-335c-45cd-841e-8dde568b62d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:33.287227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb61f6-335c-45cd-841e-8dde568b62d3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:33.287508 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb61f6-335c-45cd-841e-8dde568b62d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:33.287788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb61f6-335c-45cd-841e-8dde568b62d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:33.288585 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb61f6-335c-45cd-841e-8dde568b62d3 None admin] RBAC: Adding query filter params (role.id=d96ee0e293244ae6a04e8ce652aea68e, user.id=6972e51a7df34f1cac25e6461f876ac7, scope.system=all) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:33.289010 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb61f6-335c-45cd-841e-8dde568b62d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:33.290662 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb61f6-335c-45cd-841e-8dde568b62d3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:33.296977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 144/283] 10.222.0.62 () {56 vars in 1308 bytes} [Wed Sep 14 11:45:33 2022] GET /identity/v3/role_assignments?user.id=6972e51a7df34f1cac25e6461f876ac7&scope.system=all&role.id=d96ee0e293244ae6a04e8ce652aea68e => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:33.298104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc56cf8-831e-43e4-b119-dbd366615474 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:33.298390 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc56cf8-831e-43e4-b119-dbd366615474 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:33.298665 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc56cf8-831e-43e4-b119-dbd366615474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:33.325551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ebc56cf8-831e-43e4-b119-dbd366615474 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:33.328051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ebc56cf8-831e-43e4-b119-dbd366615474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:33.356511 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdbee3-b063-4834-9420-1b92c83fc130 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:33.415360 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8865a3c-cda9-496b-aba3-3773a4a24f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:33.415992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8865a3c-cda9-496b-aba3-3773a4a24f88 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:33.416273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8865a3c-cda9-496b-aba3-3773a4a24f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:33.416553 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8865a3c-cda9-496b-aba3-3773a4a24f88 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:33.417232 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8865a3c-cda9-496b-aba3-3773a4a24f88 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:33.417540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8865a3c-cda9-496b-aba3-3773a4a24f88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:33.421375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8865a3c-cda9-496b-aba3-3773a4a24f88 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:33.421522 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 145/284] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:33.433586 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 140/285] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:33.439974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc56cf8-831e-43e4-b119-dbd366615474 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:33.508479 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:33.547785 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc56cf8-831e-43e4-b119-dbd366615474 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:33.584144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796886e5-69e7-4145-ab5f-b224ed540b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:33.584762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796886e5-69e7-4145-ab5f-b224ed540b67 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:33.585048 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796886e5-69e7-4145-ab5f-b224ed540b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:33.585325 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796886e5-69e7-4145-ab5f-b224ed540b67 None admin] PATH_INFO: `/v3/users/c01d7e03f5e54796bc601b39c20887df` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:33.587159 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796886e5-69e7-4145-ab5f-b224ed540b67 None admin] RBAC: Authorizing `identity:get_user(user_id=c01d7e03f5e54796bc601b39c20887df)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:33.588617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796886e5-69e7-4145-ab5f-b224ed540b67 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:33.590737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 146/286] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:33 2022] GET /identity/v3/users/c01d7e03f5e54796bc601b39c20887df => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:33.596042 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8865a3c-cda9-496b-aba3-3773a4a24f88 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:33.615260 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:33.698079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796886e5-69e7-4145-ab5f-b224ed540b67 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:33.712601 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8865a3c-cda9-496b-aba3-3773a4a24f88 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:33.759729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a3377-f62a-4ae1-a82d-d88c91d8ce6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:33.760293 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a3377-f62a-4ae1-a82d-d88c91d8ce6f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:33.760575 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a3377-f62a-4ae1-a82d-d88c91d8ce6f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:33.760848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a3377-f62a-4ae1-a82d-d88c91d8ce6f None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:33.762648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a3377-f62a-4ae1-a82d-d88c91d8ce6f None admin] RBAC: Authorizing `identity:get_project(project_id=2164ed8287f243bc9a4a5347d4dc0b1d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:33.764407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a3377-f62a-4ae1-a82d-d88c91d8ce6f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:33.766675 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 141/287] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:33 2022] GET /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:33.773186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a3377-f62a-4ae1-a82d-d88c91d8ce6f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:33.845377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796886e5-69e7-4145-ab5f-b224ed540b67 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:33.878244 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:33.884506 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ddeb2-4e28-41c2-b4ba-07cdb6cfe46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:33.885593 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ddeb2-4e28-41c2-b4ba-07cdb6cfe46c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:33.886100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ddeb2-4e28-41c2-b4ba-07cdb6cfe46c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:33.892755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ddeb2-4e28-41c2-b4ba-07cdb6cfe46c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:33.898375 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ddeb2-4e28-41c2-b4ba-07cdb6cfe46c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:33.906245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-146ddeb2-4e28-41c2-b4ba-07cdb6cfe46c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:33.906245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:33.906245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:33.906245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:33.906245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:33.906245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:33.906245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:33.906245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:33.906245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:33.906245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:33.906245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:33.906245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:33.906245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:33.906245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:33.906245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:33.906245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:33.907146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:33.907146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:33.907146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:33.907146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:33.907146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:33.907146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:33.907146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:33.907146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:33.907146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:33.907146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:33.907146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:33.907146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:33.907146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:33.907146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:33.907146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:33.907146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:33.907923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:33.907923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:33.907923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:33.907923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:33.907923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:33.907923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:33.907923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:33.907923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:33.907923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:33.907923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:33.907923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:33.907923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:33.910494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 147/288] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:33.916428 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ddeb2-4e28-41c2-b4ba-07cdb6cfe46c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:33.952010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:34.000075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a3377-f62a-4ae1-a82d-d88c91d8ce6f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:34.039901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ddeb2-4e28-41c2-b4ba-07cdb6cfe46c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:34.080873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc1106-87a2-482d-b2d3-91591510cc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:34.081845 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc1106-87a2-482d-b2d3-91591510cc6e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:34.082344 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc1106-87a2-482d-b2d3-91591510cc6e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:34.082809 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc1106-87a2-482d-b2d3-91591510cc6e None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:34.083774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc1106-87a2-482d-b2d3-91591510cc6e None admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:34.084335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc1106-87a2-482d-b2d3-91591510cc6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:34.086019 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc1106-87a2-482d-b2d3-91591510cc6e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:34.095683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbe086-aa01-4102-830e-4e4b9206090d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:34.095683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbe086-aa01-4102-830e-4e4b9206090d None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:34.095683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbe086-aa01-4102-830e-4e4b9206090d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:34.095683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbe086-aa01-4102-830e-4e4b9206090d None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d/users/c01d7e03f5e54796bc601b39c20887df/roles/f792865844594321a8a67b38335cdccb` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:34.097847 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fbe086-aa01-4102-830e-4e4b9206090d None admin] RBAC: Authorizing `identity:create_grant(project_id=2164ed8287f243bc9a4a5347d4dc0b1d, user_id=c01d7e03f5e54796bc601b39c20887df, role_id=f792865844594321a8a67b38335cdccb)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:34.099453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fbe086-aa01-4102-830e-4e4b9206090d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:34.099662 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 148/289] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:34.107025 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc1106-87a2-482d-b2d3-91591510cc6e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:34.166251 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:34.192490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 142/290] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:45:33 2022] PUT /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d/users/c01d7e03f5e54796bc601b39c20887df/roles/f792865844594321a8a67b38335cdccb => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 11:45:34.225712 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc1106-87a2-482d-b2d3-91591510cc6e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:34.279496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac108a-26e8-4976-9732-e07f9cb5a9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:34.281161 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac108a-26e8-4976-9732-e07f9cb5a9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:34.281639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac108a-26e8-4976-9732-e07f9cb5a9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:34.282145 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac108a-26e8-4976-9732-e07f9cb5a9e8 None admin] PATH_INFO: `/v3/users/5b5c68ce791b4845af27773a417486fd` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:34.283581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac108a-26e8-4976-9732-e07f9cb5a9e8 None admin] RBAC: Authorizing `identity:get_user(user_id=5b5c68ce791b4845af27773a417486fd)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:34.285299 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac108a-26e8-4976-9732-e07f9cb5a9e8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:34.293218 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbe086-aa01-4102-830e-4e4b9206090d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:34.304080 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 149/291] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:34 2022] GET /identity/v3/users/5b5c68ce791b4845af27773a417486fd => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:34.420098 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbe086-aa01-4102-830e-4e4b9206090d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:34.457007 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d9c45-cd19-4b66-9f02-6e3832c76534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:34.457963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d9c45-cd19-4b66-9f02-6e3832c76534 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:34.458441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d9c45-cd19-4b66-9f02-6e3832c76534 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:34.458903 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d9c45-cd19-4b66-9f02-6e3832c76534 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:34.470926 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:34.530047 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d9c45-cd19-4b66-9f02-6e3832c76534 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:34.531831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d9c45-cd19-4b66-9f02-6e3832c76534 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:34.633592 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 143/292] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:34.638810 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac108a-26e8-4976-9732-e07f9cb5a9e8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:34.766624 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac108a-26e8-4976-9732-e07f9cb5a9e8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:34.793229 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:34.812027 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08302e0b-4fc2-4f31-a31e-10d89dc8f379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:34.812965 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08302e0b-4fc2-4f31-a31e-10d89dc8f379 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:34.813441 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08302e0b-4fc2-4f31-a31e-10d89dc8f379 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:34.813960 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08302e0b-4fc2-4f31-a31e-10d89dc8f379 None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:34.815422 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08302e0b-4fc2-4f31-a31e-10d89dc8f379 None admin] RBAC: Authorizing `identity:get_project(project_id=2164ed8287f243bc9a4a5347d4dc0b1d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:34.817127 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08302e0b-4fc2-4f31-a31e-10d89dc8f379 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:34.819269 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 150/293] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:34 2022] GET /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:34.833767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d9c45-cd19-4b66-9f02-6e3832c76534 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:34.877667 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:34.938349 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d9c45-cd19-4b66-9f02-6e3832c76534 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:34.973344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1504078-0c73-4dc0-ac70-ec3e0c7f6e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:34.973988 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1504078-0c73-4dc0-ac70-ec3e0c7f6e08 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:34.974657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1504078-0c73-4dc0-ac70-ec3e0c7f6e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:34.974929 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1504078-0c73-4dc0-ac70-ec3e0c7f6e08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:34.975709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1504078-0c73-4dc0-ac70-ec3e0c7f6e08 None admin] RBAC: Adding query filter params (role.id=d96ee0e293244ae6a04e8ce652aea68e, scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d, user.id=5b5c68ce791b4845af27773a417486fd) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:34.976022 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1504078-0c73-4dc0-ac70-ec3e0c7f6e08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:34.977440 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1504078-0c73-4dc0-ac70-ec3e0c7f6e08 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:34.984299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 144/294] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:45:34 2022] GET /identity/v3/role_assignments?user.id=5b5c68ce791b4845af27773a417486fd&scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d&role.id=d96ee0e293244ae6a04e8ce652aea68e => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:35.838283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b78ec6-6df4-4ccb-aaf1-99103bd0706e None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:35.838793 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b78ec6-6df4-4ccb-aaf1-99103bd0706e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:35.838868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b78ec6-6df4-4ccb-aaf1-99103bd0706e None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:35.840184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 151/295] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:35.846569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc0a45-0d70-4aaf-a01e-909bbe177af4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:35.847065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc0a45-0d70-4aaf-a01e-909bbe177af4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:35.847152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc0a45-0d70-4aaf-a01e-909bbe177af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:35.865279 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f2bc0a45-0d70-4aaf-a01e-909bbe177af4 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:35.866496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f2bc0a45-0d70-4aaf-a01e-909bbe177af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:35.932349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf3f06-b94d-458c-8c80-e8b61c8eef0a None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:35.932697 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf3f06-b94d-458c-8c80-e8b61c8eef0a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:35.932954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf3f06-b94d-458c-8c80-e8b61c8eef0a None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:35.934784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 152/296] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:35.937300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 145/297] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:35.940866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246602c-36d8-40ad-8357-816e8b16efa0 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:35.950527 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246602c-36d8-40ad-8357-816e8b16efa0 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:35.950527 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246602c-36d8-40ad-8357-816e8b16efa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:35.962439 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8246602c-36d8-40ad-8357-816e8b16efa0 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:35.963681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8246602c-36d8-40ad-8357-816e8b16efa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:36.033466 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 153/298] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:36.062915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab4cfa-2caa-4bd2-9e39-ccce8d5ba369 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:36.063522 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab4cfa-2caa-4bd2-9e39-ccce8d5ba369 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:36.063998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab4cfa-2caa-4bd2-9e39-ccce8d5ba369 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:36.065791 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 146/299] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:36.070995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27429617-e9e2-49c6-8997-1acbcbf2f8b0 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:36.071285 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27429617-e9e2-49c6-8997-1acbcbf2f8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:36.071810 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27429617-e9e2-49c6-8997-1acbcbf2f8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:36.090469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-27429617-e9e2-49c6-8997-1acbcbf2f8b0 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:36.091659 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-27429617-e9e2-49c6-8997-1acbcbf2f8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:36.157437 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 154/300] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:36.224187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab4cfa-2caa-4bd2-9e39-ccce8d5ba369 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:36.260110 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:36.326158 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab4cfa-2caa-4bd2-9e39-ccce8d5ba369 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:36.337127 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27429617-e9e2-49c6-8997-1acbcbf2f8b0 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:36.364358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396efe5-63e8-4aaf-814f-7322cdf54dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:36.365711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396efe5-63e8-4aaf-814f-7322cdf54dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:36.366356 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396efe5-63e8-4aaf-814f-7322cdf54dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:36.366874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396efe5-63e8-4aaf-814f-7322cdf54dd5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:36.378768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396efe5-63e8-4aaf-814f-7322cdf54dd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:36.384831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-7396efe5-63e8-4aaf-814f-7322cdf54dd5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:36.384831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:36.384831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:36.384831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:36.384831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:36.384831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:36.384831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:36.384831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:36.384831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:36.384831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:36.384831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:36.384831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:36.384831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:36.384831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:36.384831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:36.384831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:36.385927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:36.385927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:36.385927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:36.385927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:36.385927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:36.385927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:36.385927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:36.385927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:36.385927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:36.385927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:36.385927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:36.385927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:36.385927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:36.385927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:36.385927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:36.385927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:36.386909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:36.386909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:36.386909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:36.386909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:36.386909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:36.386909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:36.386909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:36.386909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:36.386909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:36.386909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:36.386909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:36.386909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:36.394184 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 147/301] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:36.395034 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396efe5-63e8-4aaf-814f-7322cdf54dd5 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:36.404663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:36.467728 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27429617-e9e2-49c6-8997-1acbcbf2f8b0 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:36.496822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:36.504121 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2268019-f86f-4f58-a105-0dc78be5175a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:36.504303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396efe5-63e8-4aaf-814f-7322cdf54dd5 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:36.504760 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2268019-f86f-4f58-a105-0dc78be5175a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:36.505051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2268019-f86f-4f58-a105-0dc78be5175a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:36.505328 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2268019-f86f-4f58-a105-0dc78be5175a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:36.518580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2268019-f86f-4f58-a105-0dc78be5175a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:36.523423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-e2268019-f86f-4f58-a105-0dc78be5175a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 11:45:36.523423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:36.523423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:36.523423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:36.523423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:36.523423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:36.523423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:36.523423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:36.523423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:36.523423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:36.523423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:36.523423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:36.523423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:36.523423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:36.523423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:36.523423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:36.524296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:36.524296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:36.524296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:36.524296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:36.524296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:36.524296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:36.524296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:36.524296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:36.524296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:36.524296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:36.524296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:36.524296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:36.524296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:36.524296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:36.524296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:36.524296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:36.525770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:36.525770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:36.525770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:36.525770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:36.525770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:36.525770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:36.525770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:36.525770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:36.525770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:36.525770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:36.525770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 11:45:36.525770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:36.525770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 155/302] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:36.525770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2268019-f86f-4f58-a105-0dc78be5175a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:36.543271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37ffd3-aa85-4eef-a85c-01cd0fb396ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:36.543881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37ffd3-aa85-4eef-a85c-01cd0fb396ae None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:36.544163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37ffd3-aa85-4eef-a85c-01cd0fb396ae None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:36.544439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37ffd3-aa85-4eef-a85c-01cd0fb396ae None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:36.545096 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37ffd3-aa85-4eef-a85c-01cd0fb396ae None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:36.546117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37ffd3-aa85-4eef-a85c-01cd0fb396ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:36.547795 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37ffd3-aa85-4eef-a85c-01cd0fb396ae None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:36.557623 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 148/303] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:36.558933 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37ffd3-aa85-4eef-a85c-01cd0fb396ae None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:36.633960 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2268019-f86f-4f58-a105-0dc78be5175a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:36.644323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:36.668415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c586d7-b0a4-44e8-9f20-b29f6948cb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:36.669038 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c586d7-b0a4-44e8-9f20-b29f6948cb85 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:36.669318 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c586d7-b0a4-44e8-9f20-b29f6948cb85 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:36.669588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c586d7-b0a4-44e8-9f20-b29f6948cb85 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:36.671498 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c586d7-b0a4-44e8-9f20-b29f6948cb85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:36.673909 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c586d7-b0a4-44e8-9f20-b29f6948cb85 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:36.674281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37ffd3-aa85-4eef-a85c-01cd0fb396ae None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:36.680752 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 156/304] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:45:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:36.680752 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c586d7-b0a4-44e8-9f20-b29f6948cb85 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:36.711059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46aa968-69dc-45b8-a89d-6455c6730c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:36.712083 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46aa968-69dc-45b8-a89d-6455c6730c01 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:36.712368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46aa968-69dc-45b8-a89d-6455c6730c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:36.712795 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46aa968-69dc-45b8-a89d-6455c6730c01 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:36.713483 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46aa968-69dc-45b8-a89d-6455c6730c01 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:36.713898 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46aa968-69dc-45b8-a89d-6455c6730c01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:36.715249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46aa968-69dc-45b8-a89d-6455c6730c01 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:36.725394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 149/305] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:36.726078 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46aa968-69dc-45b8-a89d-6455c6730c01 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:36.747126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:36.796446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c586d7-b0a4-44e8-9f20-b29f6948cb85 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:36.832766 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4f673-578a-4895-b3e0-31ef56e3bf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:36.833404 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4f673-578a-4895-b3e0-31ef56e3bf5a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:36.833686 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4f673-578a-4895-b3e0-31ef56e3bf5a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:36.833990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4f673-578a-4895-b3e0-31ef56e3bf5a None admin] PATH_INFO: `/v3/users/6972e51a7df34f1cac25e6461f876ac7` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:36.835110 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4f673-578a-4895-b3e0-31ef56e3bf5a None admin] RBAC: Authorizing `identity:get_user(user_id=6972e51a7df34f1cac25e6461f876ac7)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:36.835549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46aa968-69dc-45b8-a89d-6455c6730c01 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:36.836558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4f673-578a-4895-b3e0-31ef56e3bf5a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:36.838158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 157/306] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:36 2022] GET /identity/v3/users/6972e51a7df34f1cac25e6461f876ac7 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:36.838921 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4f673-578a-4895-b3e0-31ef56e3bf5a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:36.868906 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b937367-0649-4a9f-afba-ca309292d281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:36.869506 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b937367-0649-4a9f-afba-ca309292d281 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:36.869785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b937367-0649-4a9f-afba-ca309292d281 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:36.870082 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b937367-0649-4a9f-afba-ca309292d281 None admin] PATH_INFO: `/v3/groups` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:36.870702 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b937367-0649-4a9f-afba-ca309292d281 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:36.872124 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b937367-0649-4a9f-afba-ca309292d281 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:36.895933 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:36.936366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4f673-578a-4895-b3e0-31ef56e3bf5a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:36.958136 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 150/307] 10.222.0.62 () {60 vars in 1134 bytes} [Wed Sep 14 11:45:36 2022] POST /identity/v3/groups => generated 229 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:45:36.959636 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f36d8-1feb-4306-b662-155bf0cc4274 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:36.959929 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f36d8-1feb-4306-b662-155bf0cc4274 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:36.960245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f36d8-1feb-4306-b662-155bf0cc4274 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:36.961517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 151/308] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:36.962257 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f36d8-1feb-4306-b662-155bf0cc4274 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:36.983134 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5982d6-2a32-4147-8027-655d278c05a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:36.983801 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5982d6-2a32-4147-8027-655d278c05a3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:36.984095 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5982d6-2a32-4147-8027-655d278c05a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:36.984378 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5982d6-2a32-4147-8027-655d278c05a3 None admin] PATH_INFO: `/v3/users/09a76415895543358b3a19da03c199c4` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:36.999955 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5982d6-2a32-4147-8027-655d278c05a3 None admin] RBAC: Authorizing `identity:get_user(user_id=09a76415895543358b3a19da03c199c4)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:37.001457 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5982d6-2a32-4147-8027-655d278c05a3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:37.003206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 158/309] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:36 2022] GET /identity/v3/users/09a76415895543358b3a19da03c199c4 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:37.004403 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe837a-6778-4b51-9c8c-003abf4ec9a7 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:37.004698 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe837a-6778-4b51-9c8c-003abf4ec9a7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:37.004981 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe837a-6778-4b51-9c8c-003abf4ec9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:37.024471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3efe837a-6778-4b51-9c8c-003abf4ec9a7 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:37.025726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3efe837a-6778-4b51-9c8c-003abf4ec9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:37.066512 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:37.090676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f36d8-1feb-4306-b662-155bf0cc4274 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:37.110398 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 159/310] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:37.111442 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe837a-6778-4b51-9c8c-003abf4ec9a7 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:37.133754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d557e6-9ae5-4309-aeb0-f65969345e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:37.133754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d557e6-9ae5-4309-aeb0-f65969345e21 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:37.133754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d557e6-9ae5-4309-aeb0-f65969345e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:37.133754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d557e6-9ae5-4309-aeb0-f65969345e21 None admin] PATH_INFO: `/v3/system/users/6972e51a7df34f1cac25e6461f876ac7/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:37.136418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d557e6-9ae5-4309-aeb0-f65969345e21 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6972e51a7df34f1cac25e6461f876ac7, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:37.137899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d557e6-9ae5-4309-aeb0-f65969345e21 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:37.153877 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:37.198643 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 152/311] 10.222.0.62 () {58 vars in 1334 bytes} [Wed Sep 14 11:45:36 2022] PUT /identity/v3/system/users/6972e51a7df34f1cac25e6461f876ac7/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:45:37.225845 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe837a-6778-4b51-9c8c-003abf4ec9a7 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:37.270222 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35b440-afea-4473-8329-e2eaf78269b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:37.270943 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35b440-afea-4473-8329-e2eaf78269b3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:37.271229 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35b440-afea-4473-8329-e2eaf78269b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:37.271506 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35b440-afea-4473-8329-e2eaf78269b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:37.272316 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f35b440-afea-4473-8329-e2eaf78269b3 None admin] RBAC: Adding query filter params (role.id=f792865844594321a8a67b38335cdccb, user.id=09a76415895543358b3a19da03c199c4, scope.system=all) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:37.272632 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f35b440-afea-4473-8329-e2eaf78269b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:37.274130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f35b440-afea-4473-8329-e2eaf78269b3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:37.280525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 160/312] 10.222.0.62 () {56 vars in 1308 bytes} [Wed Sep 14 11:45:37 2022] GET /identity/v3/role_assignments?user.id=09a76415895543358b3a19da03c199c4&scope.system=all&role.id=f792865844594321a8a67b38335cdccb => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:37.383687 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d557e6-9ae5-4309-aeb0-f65969345e21 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:37.496088 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d557e6-9ae5-4309-aeb0-f65969345e21 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:37.532881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7deafe-1d11-4b92-bde4-3a934b6c84eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:37.533525 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7deafe-1d11-4b92-bde4-3a934b6c84eb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:37.533809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7deafe-1d11-4b92-bde4-3a934b6c84eb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:37.534113 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7deafe-1d11-4b92-bde4-3a934b6c84eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:37.544763 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7deafe-1d11-4b92-bde4-3a934b6c84eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:37.550983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-cc7deafe-1d11-4b92-bde4-3a934b6c84eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 11:45:37.550983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:37.550983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:37.550983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:37.550983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:37.550983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:37.550983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:37.550983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:37.550983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:37.550983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:37.550983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:37.550983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:37.550983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:37.550983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:37.550983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:37.550983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:37.551920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:37.551920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:37.551920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:37.551920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:37.551920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:37.551920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:37.551920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:37.551920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:37.551920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:37.551920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:37.551920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:37.551920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:37.551920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:37.551920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:37.551920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:37.551920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:37.552717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:37.552717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:37.552717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:37.552717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:37.552717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:37.552717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:37.552717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:37.552717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:37.552717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:37.552717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:37.552717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 11:45:37.552717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:37.552717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 153/313] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:37.557947 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35b440-afea-4473-8329-e2eaf78269b3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:37.662780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:37.669114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35b440-afea-4473-8329-e2eaf78269b3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:37.706744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6e9fd-34f6-4d76-b1a6-c25e1bc90315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:37.707360 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6e9fd-34f6-4d76-b1a6-c25e1bc90315 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:37.707645 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6e9fd-34f6-4d76-b1a6-c25e1bc90315 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:37.707925 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6e9fd-34f6-4d76-b1a6-c25e1bc90315 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:37.708605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6e9fd-34f6-4d76-b1a6-c25e1bc90315 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:37.708913 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6e9fd-34f6-4d76-b1a6-c25e1bc90315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:37.710335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6e9fd-34f6-4d76-b1a6-c25e1bc90315 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:37.722024 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 161/314] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:37.727445 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7deafe-1d11-4b92-bde4-3a934b6c84eb None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:37.730411 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:37.847591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7deafe-1d11-4b92-bde4-3a934b6c84eb None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:37.883134 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542a13d-5723-4d23-abf5-a6008a5ce574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:37.884562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542a13d-5723-4d23-abf5-a6008a5ce574 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:37.885053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542a13d-5723-4d23-abf5-a6008a5ce574 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:37.885530 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542a13d-5723-4d23-abf5-a6008a5ce574 None admin] PATH_INFO: `/v3/users/c01d7e03f5e54796bc601b39c20887df` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:37.886922 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542a13d-5723-4d23-abf5-a6008a5ce574 None admin] RBAC: Authorizing `identity:get_user(user_id=c01d7e03f5e54796bc601b39c20887df)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:37.888582 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542a13d-5723-4d23-abf5-a6008a5ce574 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:37.891351 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 154/315] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:37 2022] GET /identity/v3/users/c01d7e03f5e54796bc601b39c20887df => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:37.899374 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6e9fd-34f6-4d76-b1a6-c25e1bc90315 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:38.047315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6e9fd-34f6-4d76-b1a6-c25e1bc90315 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:38.083133 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d5f94-6dd5-4124-8f18-60cbf81c41c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:38.083661 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d5f94-6dd5-4124-8f18-60cbf81c41c3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:38.083942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d5f94-6dd5-4124-8f18-60cbf81c41c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:38.084213 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d5f94-6dd5-4124-8f18-60cbf81c41c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:38.112507 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:38.153648 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d5f94-6dd5-4124-8f18-60cbf81c41c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:38.155802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d5f94-6dd5-4124-8f18-60cbf81c41c3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:38.266875 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 162/316] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:38.272238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542a13d-5723-4d23-abf5-a6008a5ce574 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:38.348320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:38.383175 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542a13d-5723-4d23-abf5-a6008a5ce574 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:38.419145 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2eb27d-0fec-49be-9862-7019d74324fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:38.420053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2eb27d-0fec-49be-9862-7019d74324fe None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:38.421045 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2eb27d-0fec-49be-9862-7019d74324fe None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:38.421498 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2eb27d-0fec-49be-9862-7019d74324fe None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:38.422855 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2eb27d-0fec-49be-9862-7019d74324fe None admin] RBAC: Authorizing `identity:get_project(project_id=2164ed8287f243bc9a4a5347d4dc0b1d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:38.424470 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2eb27d-0fec-49be-9862-7019d74324fe None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:38.426585 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 155/317] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:38 2022] GET /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:38.432812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d5f94-6dd5-4124-8f18-60cbf81c41c3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:38.493845 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:38.541632 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d5f94-6dd5-4124-8f18-60cbf81c41c3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:38.577444 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851eba5b-b139-431c-9d71-9c5e4a29e183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:38.578073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851eba5b-b139-431c-9d71-9c5e4a29e183 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:38.578351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851eba5b-b139-431c-9d71-9c5e4a29e183 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:38.578624 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851eba5b-b139-431c-9d71-9c5e4a29e183 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:38.579360 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851eba5b-b139-431c-9d71-9c5e4a29e183 None admin] RBAC: Adding query filter params (role.id=f792865844594321a8a67b38335cdccb, scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d, user.id=c01d7e03f5e54796bc601b39c20887df) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:38.579651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851eba5b-b139-431c-9d71-9c5e4a29e183 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:38.581095 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851eba5b-b139-431c-9d71-9c5e4a29e183 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:38.587103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 163/318] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:45:38 2022] GET /identity/v3/role_assignments?user.id=c01d7e03f5e54796bc601b39c20887df&scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d&role.id=f792865844594321a8a67b38335cdccb => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:39.253770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e39dc-cd2a-49de-88aa-830eb4f77e57 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:39.254328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e39dc-cd2a-49de-88aa-830eb4f77e57 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:39.254404 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e39dc-cd2a-49de-88aa-830eb4f77e57 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:39.255838 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 156/319] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:39.261756 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873560c-7adf-4681-8cae-a2c77ec0b997 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:39.262204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873560c-7adf-4681-8cae-a2c77ec0b997 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:39.262379 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873560c-7adf-4681-8cae-a2c77ec0b997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:39.281024 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a873560c-7adf-4681-8cae-a2c77ec0b997 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:39.282246 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a873560c-7adf-4681-8cae-a2c77ec0b997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:39.350404 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 164/320] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:39.506028 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1120945-594a-489c-a3f0-e06a4a165933 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:39.506334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1120945-594a-489c-a3f0-e06a4a165933 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:39.506507 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1120945-594a-489c-a3f0-e06a4a165933 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:39.507992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 157/321] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:39.513200 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6d82a-5825-4fee-bee2-55058715d8a1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:39.513496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6d82a-5825-4fee-bee2-55058715d8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:39.513773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6d82a-5825-4fee-bee2-55058715d8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:39.530747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dcb87-6579-4821-8ce9-a2f612cf9b3b None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:39.531036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dcb87-6579-4821-8ce9-a2f612cf9b3b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:39.531317 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dcb87-6579-4821-8ce9-a2f612cf9b3b None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:39.531811 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-eff6d82a-5825-4fee-bee2-55058715d8a1 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:39.532760 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 158/322] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:45:39.533046 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-eff6d82a-5825-4fee-bee2-55058715d8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:39.537329 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a531c33-b324-4890-9084-55a47cfbdfb8 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:39.537615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a531c33-b324-4890-9084-55a47cfbdfb8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:39.537919 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a531c33-b324-4890-9084-55a47cfbdfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:39.557450 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0a531c33-b324-4890-9084-55a47cfbdfb8 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:39.558676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0a531c33-b324-4890-9084-55a47cfbdfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:39.582114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:39.602229 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 165/323] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:39.623184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6d82a-5825-4fee-bee2-55058715d8a1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:39.629898 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 159/324] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:45:39.728888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6d82a-5825-4fee-bee2-55058715d8a1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:39.763704 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e514b-45df-4632-bb97-a26469f6dbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:39.764634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e514b-45df-4632-bb97-a26469f6dbc3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:39.765127 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e514b-45df-4632-bb97-a26469f6dbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:39.765600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e514b-45df-4632-bb97-a26469f6dbc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:39.766975 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e514b-45df-4632-bb97-a26469f6dbc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:39.768672 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e514b-45df-4632-bb97-a26469f6dbc3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:39.770791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 166/325] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:45:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:39.776012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a531c33-b324-4890-9084-55a47cfbdfb8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:39.849902 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:39.871799 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e514b-45df-4632-bb97-a26469f6dbc3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:39.883622 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a531c33-b324-4890-9084-55a47cfbdfb8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:39.917684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5de2b5-4f69-478a-b0ae-c4504cbcc7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:39.918314 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5de2b5-4f69-478a-b0ae-c4504cbcc7a1 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:39.918601 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5de2b5-4f69-478a-b0ae-c4504cbcc7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:39.918877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5de2b5-4f69-478a-b0ae-c4504cbcc7a1 None admin] PATH_INFO: `/v3/groups` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:39.919517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5de2b5-4f69-478a-b0ae-c4504cbcc7a1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:39.921515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5de2b5-4f69-478a-b0ae-c4504cbcc7a1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:39.926468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:39.976447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e514b-45df-4632-bb97-a26469f6dbc3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:40.010410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d985cd-5edf-4305-b352-13d2da1fbd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:40.011047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d985cd-5edf-4305-b352-13d2da1fbd77 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:40.011331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d985cd-5edf-4305-b352-13d2da1fbd77 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:40.011610 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d985cd-5edf-4305-b352-13d2da1fbd77 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:40.015027 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 160/326] 10.222.0.62 () {60 vars in 1134 bytes} [Wed Sep 14 11:45:39 2022] POST /identity/v3/groups => generated 226 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 11:45:40.015795 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5de2b5-4f69-478a-b0ae-c4504cbcc7a1 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:40.029179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d985cd-5edf-4305-b352-13d2da1fbd77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:40.034896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-89d985cd-5edf-4305-b352-13d2da1fbd77 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:40.034896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:40.034896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:40.034896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:40.034896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:40.034896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:40.034896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:40.034896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:40.034896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:40.034896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:40.034896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:40.034896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:40.034896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:40.034896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:40.034896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:40.034896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:40.036079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:40.036079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:40.036079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:40.036079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:40.036079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:40.036079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:40.036079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:40.036079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:40.036079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:40.036079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:40.036079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:40.036079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:40.036079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:40.036079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:40.036079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:40.036079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:40.037654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:40.037654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:40.037654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:40.037654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:40.037654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:40.037654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:40.037654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:40.037654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:40.037654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:40.037654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:40.037654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:40.037654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:40.037654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 167/327] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:45:40.042343 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d985cd-5edf-4305-b352-13d2da1fbd77 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:40.142933 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5de2b5-4f69-478a-b0ae-c4504cbcc7a1 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:40.165736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d985cd-5edf-4305-b352-13d2da1fbd77 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:40.173658 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:40.178643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:40.184344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b87386-53a7-433f-b570-9941a129cc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:40.185027 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b87386-53a7-433f-b570-9941a129cc75 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:40.185267 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b87386-53a7-433f-b570-9941a129cc75 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:40.185539 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b87386-53a7-433f-b570-9941a129cc75 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:40.196779 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b87386-53a7-433f-b570-9941a129cc75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:40.202336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-f3b87386-53a7-433f-b570-9941a129cc75 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 11:45:40.202336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:40.202336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:40.202336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:40.202336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:40.202336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:40.202336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:40.202336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:40.202336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:40.202336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:40.202336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:40.202336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:40.202336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:40.202336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:40.202336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:40.202336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:40.203458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:40.203458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:40.203458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:40.203458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:40.203458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:40.203458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:40.203458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:40.203458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:40.203458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:40.203458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:40.203458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:40.203458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:40.203458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:40.203458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:40.203458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:40.203458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:40.205347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:40.205347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:40.205347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:40.205347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:40.205347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:40.205347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:40.205347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:40.205347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:40.205347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:40.205347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:40.205347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 11:45:40.205347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:40.205347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 161/328] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:40.207182 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c0e18-22e5-4ce4-8b1a-2bd1cdaa0f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:40.208078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c0e18-22e5-4ce4-8b1a-2bd1cdaa0f04 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:40.208371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c0e18-22e5-4ce4-8b1a-2bd1cdaa0f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:40.208680 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c0e18-22e5-4ce4-8b1a-2bd1cdaa0f04 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:40.209115 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b87386-53a7-433f-b570-9941a129cc75 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:40.209356 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c0e18-22e5-4ce4-8b1a-2bd1cdaa0f04 None admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:40.209667 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c0e18-22e5-4ce4-8b1a-2bd1cdaa0f04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:40.211167 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c0e18-22e5-4ce4-8b1a-2bd1cdaa0f04 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:40.224271 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 168/329] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:40.230545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c0e18-22e5-4ce4-8b1a-2bd1cdaa0f04 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:40.331305 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b87386-53a7-433f-b570-9941a129cc75 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:40.354327 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c0e18-22e5-4ce4-8b1a-2bd1cdaa0f04 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:40.369903 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ce26a-8058-4885-af57-2345dc60357e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:40.370612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ce26a-8058-4885-af57-2345dc60357e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:40.371054 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ce26a-8058-4885-af57-2345dc60357e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:40.371299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ce26a-8058-4885-af57-2345dc60357e None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:40.372034 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ce26a-8058-4885-af57-2345dc60357e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:40.372407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ce26a-8058-4885-af57-2345dc60357e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:40.373907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ce26a-8058-4885-af57-2345dc60357e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:40.385601 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 162/330] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:40.392240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ce26a-8058-4885-af57-2345dc60357e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:40.402004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446be91c-2d2d-4b73-a597-07dd68ff311e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:40.402650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446be91c-2d2d-4b73-a597-07dd68ff311e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:40.402934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446be91c-2d2d-4b73-a597-07dd68ff311e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:40.403238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446be91c-2d2d-4b73-a597-07dd68ff311e None admin] PATH_INFO: `/v3/users/6972e51a7df34f1cac25e6461f876ac7` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:40.404358 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446be91c-2d2d-4b73-a597-07dd68ff311e None admin] RBAC: Authorizing `identity:get_user(user_id=6972e51a7df34f1cac25e6461f876ac7)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:40.406659 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446be91c-2d2d-4b73-a597-07dd68ff311e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:40.408440 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 169/331] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:40 2022] GET /identity/v3/users/6972e51a7df34f1cac25e6461f876ac7 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:40.414307 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446be91c-2d2d-4b73-a597-07dd68ff311e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:40.454793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:40.464974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:40.515605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ce26a-8058-4885-af57-2345dc60357e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:40.537192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446be91c-2d2d-4b73-a597-07dd68ff311e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:40.557047 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19691ef5-5ad5-46c5-abc7-bbe333633443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:40.558273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19691ef5-5ad5-46c5-abc7-bbe333633443 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:40.558710 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19691ef5-5ad5-46c5-abc7-bbe333633443 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:40.559130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19691ef5-5ad5-46c5-abc7-bbe333633443 None admin] PATH_INFO: `/v3/users/09a76415895543358b3a19da03c199c4` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:40.560467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19691ef5-5ad5-46c5-abc7-bbe333633443 None admin] RBAC: Authorizing `identity:get_user(user_id=09a76415895543358b3a19da03c199c4)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:40.563216 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19691ef5-5ad5-46c5-abc7-bbe333633443 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:40.565006 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 163/332] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:40 2022] GET /identity/v3/users/09a76415895543358b3a19da03c199c4 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:40.571477 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19691ef5-5ad5-46c5-abc7-bbe333633443 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:40.582435 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4da41-cc9e-420a-91b8-7a7715df108a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:40.583121 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4da41-cc9e-420a-91b8-7a7715df108a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:40.583509 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4da41-cc9e-420a-91b8-7a7715df108a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:40.583887 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4da41-cc9e-420a-91b8-7a7715df108a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:40.584789 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d4da41-cc9e-420a-91b8-7a7715df108a None admin] RBAC: Adding query filter params (role.id=d96ee0e293244ae6a04e8ce652aea68e, user.id=6972e51a7df34f1cac25e6461f876ac7, scope.system=all) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:40.585205 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d4da41-cc9e-420a-91b8-7a7715df108a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:40.586872 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d4da41-cc9e-420a-91b8-7a7715df108a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:40.599248 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 170/333] 10.222.0.62 () {56 vars in 1308 bytes} [Wed Sep 14 11:45:40 2022] GET /identity/v3/role_assignments?user.id=6972e51a7df34f1cac25e6461f876ac7&scope.system=all&role.id=d96ee0e293244ae6a04e8ce652aea68e => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:40.692134 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19691ef5-5ad5-46c5-abc7-bbe333633443 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:40.720711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:40.739712 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c138bac-de6a-4c4b-ac2c-4aaec5c2334a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:40.740386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c138bac-de6a-4c4b-ac2c-4aaec5c2334a None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:40.740668 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c138bac-de6a-4c4b-ac2c-4aaec5c2334a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:40.740946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c138bac-de6a-4c4b-ac2c-4aaec5c2334a None admin] PATH_INFO: `/v3/system/users/09a76415895543358b3a19da03c199c4/roles/f792865844594321a8a67b38335cdccb` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:40.747247 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c138bac-de6a-4c4b-ac2c-4aaec5c2334a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=09a76415895543358b3a19da03c199c4, role_id=f792865844594321a8a67b38335cdccb)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:40.747247 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c138bac-de6a-4c4b-ac2c-4aaec5c2334a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:40.821754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 164/334] 10.222.0.62 () {58 vars in 1334 bytes} [Wed Sep 14 11:45:40 2022] PUT /identity/v3/system/users/09a76415895543358b3a19da03c199c4/roles/f792865844594321a8a67b38335cdccb => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:45:42.522661 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb10d6-8a77-4337-a6a0-a86ea7b68bc3 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:42.523248 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb10d6-8a77-4337-a6a0-a86ea7b68bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:42.523248 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb10d6-8a77-4337-a6a0-a86ea7b68bc3 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:42.524924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 171/335] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:42.530424 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac9f4c-dd70-4e4d-a5d4-7f2bfed52377 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:42.530887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac9f4c-dd70-4e4d-a5d4-7f2bfed52377 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:42.531184 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac9f4c-dd70-4e4d-a5d4-7f2bfed52377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:42.558930 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a7ac9f4c-dd70-4e4d-a5d4-7f2bfed52377 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:42.560162 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a7ac9f4c-dd70-4e4d-a5d4-7f2bfed52377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:42.648982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 165/336] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:43.190281 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a545ba-5ec1-4df8-b862-2d08334553b1 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:43.190635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a545ba-5ec1-4df8-b862-2d08334553b1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:43.190876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a545ba-5ec1-4df8-b862-2d08334553b1 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:43.193821 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f839bca-4db9-4c19-aa9b-a6a18cd5447a None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:43.194144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f839bca-4db9-4c19-aa9b-a6a18cd5447a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:43.198245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f839bca-4db9-4c19-aa9b-a6a18cd5447a None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:43.198245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 166/337] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:43.198416 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 172/338] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:43.198493 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f839bca-4db9-4c19-aa9b-a6a18cd5447a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:43.201977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8259b90-e505-410f-9729-2b1b0e7cfb24 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:43.202716 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8259b90-e505-410f-9729-2b1b0e7cfb24 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:43.203097 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8259b90-e505-410f-9729-2b1b0e7cfb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:43.231793 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b8259b90-e505-410f-9729-2b1b0e7cfb24 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:43.233683 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b8259b90-e505-410f-9729-2b1b0e7cfb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:43.369425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 173/339] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:43.371942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e43677-4ec4-4a52-90cf-e4f9052cb31e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:43.372244 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e43677-4ec4-4a52-90cf-e4f9052cb31e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:43.372531 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e43677-4ec4-4a52-90cf-e4f9052cb31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:43.386304 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f839bca-4db9-4c19-aa9b-a6a18cd5447a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:43.403412 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:43.404187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c0e43677-4ec4-4a52-90cf-e4f9052cb31e None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:43.405450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c0e43677-4ec4-4a52-90cf-e4f9052cb31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:43.427651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a87aa-e297-442a-af7f-9def48f9f2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:43.428329 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a87aa-e297-442a-af7f-9def48f9f2be None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:43.428700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a87aa-e297-442a-af7f-9def48f9f2be None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:43.428970 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a87aa-e297-442a-af7f-9def48f9f2be None admin] PATH_INFO: `/v3/roles/member` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:43.440720 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a87aa-e297-442a-af7f-9def48f9f2be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:43.444677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:43.446593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-911a87aa-e297-442a-af7f-9def48f9f2be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:43.446593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:43.446593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:43.446593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:43.446593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:43.446593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:43.446593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:43.446593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:43.446593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:43.446593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:43.446593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:43.446593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:43.446593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:43.446593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:43.446593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:43.446593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:43.447499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:43.447499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:43.447499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:43.447499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:43.447499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:43.447499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:43.447499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:43.447499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:43.447499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:43.447499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:43.447499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:43.447499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:43.447499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:43.447499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:43.447499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:43.447499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:43.448433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:43.448433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:43.448433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:43.448433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:43.448433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:43.448433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:43.448433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:43.448433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:43.448433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:43.448433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:43.448433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:43.448433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:43.448433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 167/340] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:43.456802 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a87aa-e297-442a-af7f-9def48f9f2be None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:43.519493 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 174/341] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:43.634009 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a87aa-e297-442a-af7f-9def48f9f2be None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:43.674080 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34e430-1e54-450f-bb0d-c134c6228873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:43.674695 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34e430-1e54-450f-bb0d-c134c6228873 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:43.675233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34e430-1e54-450f-bb0d-c134c6228873 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:43.675513 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34e430-1e54-450f-bb0d-c134c6228873 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:43.676184 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34e430-1e54-450f-bb0d-c134c6228873 None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:43.676544 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34e430-1e54-450f-bb0d-c134c6228873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:43.677952 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34e430-1e54-450f-bb0d-c134c6228873 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:43.687991 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 168/342] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:43.693420 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e43677-4ec4-4a52-90cf-e4f9052cb31e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:43.837560 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e43677-4ec4-4a52-90cf-e4f9052cb31e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:43.892004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b9732-c367-484d-bf3c-a3215fe0adea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:43.892646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b9732-c367-484d-bf3c-a3215fe0adea None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:43.892934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b9732-c367-484d-bf3c-a3215fe0adea None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:43.893286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b9732-c367-484d-bf3c-a3215fe0adea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:43.928230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:43.939586 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34e430-1e54-450f-bb0d-c134c6228873 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:44.010334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:44.016044 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b9732-c367-484d-bf3c-a3215fe0adea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:44.019426 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b9732-c367-484d-bf3c-a3215fe0adea None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:44.117225 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34e430-1e54-450f-bb0d-c134c6228873 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:44.156252 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc102dc-5966-42da-b05f-4f8d1bc3cd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:44.156589 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 175/343] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:44.156967 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc102dc-5966-42da-b05f-4f8d1bc3cd53 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:44.157257 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc102dc-5966-42da-b05f-4f8d1bc3cd53 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:44.157531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc102dc-5966-42da-b05f-4f8d1bc3cd53 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:44.157922 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd12de3-d0cf-4067-98a8-a20f3905bba2 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:44.158213 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd12de3-d0cf-4067-98a8-a20f3905bba2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:44.158493 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd12de3-d0cf-4067-98a8-a20f3905bba2 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:44.164519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 176/344] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:44.165534 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd12de3-d0cf-4067-98a8-a20f3905bba2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:44.170548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc102dc-5966-42da-b05f-4f8d1bc3cd53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:44.175815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-7fc102dc-5966-42da-b05f-4f8d1bc3cd53 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:44.175815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:44.175815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:44.175815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:44.175815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:44.175815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:44.175815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:44.175815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:44.175815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:44.175815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:44.175815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:44.175815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:44.175815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:44.175815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:44.175815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:44.175815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:44.176704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:44.176704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:44.176704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:44.176704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:44.176704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:44.176704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:44.176704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:44.176704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:44.176704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:44.176704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:44.176704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:44.176704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:44.176704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:44.176704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:44.176704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:44.176704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:44.177503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:44.177503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:44.177503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:44.177503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:44.177503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:44.177503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:44.177503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:44.177503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:44.177503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:44.177503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:44.177503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:44.177503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:44.177503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 169/345] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:45:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:45:44.178232 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc102dc-5966-42da-b05f-4f8d1bc3cd53 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:44.240932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:44.292263 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:44.301435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc102dc-5966-42da-b05f-4f8d1bc3cd53 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:44.303212 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd12de3-d0cf-4067-98a8-a20f3905bba2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:44.348654 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c87fb-7325-4dc3-8ffa-4e4e831c1775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:44.348893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd23623-d1b9-43cc-ac00-0242903a7ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:44.349300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c87fb-7325-4dc3-8ffa-4e4e831c1775 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:44.349625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c87fb-7325-4dc3-8ffa-4e4e831c1775 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:44.350041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c87fb-7325-4dc3-8ffa-4e4e831c1775 None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:44.350335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd23623-d1b9-43cc-ac00-0242903a7ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:44.350801 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd23623-d1b9-43cc-ac00-0242903a7ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:44.351195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c87fb-7325-4dc3-8ffa-4e4e831c1775 None admin] RBAC: Authorizing `identity:get_project(project_id=19b75bd757df4c8cac09dd67b1f0ca8d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:44.351378 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd23623-d1b9-43cc-ac00-0242903a7ad6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:44.352644 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c87fb-7325-4dc3-8ffa-4e4e831c1775 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:44.354549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 170/346] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:44 2022] GET /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:44.356149 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad166a-75a5-4141-9ca5-0ade6a4ce5f9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:44.356441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad166a-75a5-4141-9ca5-0ade6a4ce5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:44.356722 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad166a-75a5-4141-9ca5-0ade6a4ce5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:44.365269 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd23623-d1b9-43cc-ac00-0242903a7ad6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:44.372658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-9dd23623-d1b9-43cc-ac00-0242903a7ad6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:44.372658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:44.372658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:44.372658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:44.372658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:44.372658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:44.372658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:44.372658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:44.372658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:44.372658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:44.372658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:44.372658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:44.372658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:44.372658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:44.372658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:44.372658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:44.374411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:44.374411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:44.374411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:44.374411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:44.374411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:44.374411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:44.374411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:44.374411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:44.374411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:44.374411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:44.374411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:44.374411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:44.374411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:44.374411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:44.374411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:44.374411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:44.375412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:44.375412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:44.375412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:44.375412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:44.375412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:44.375412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:44.375412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:44.375412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:44.375412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:44.375412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:44.375412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:44.375412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:44.377676 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 177/347] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:44.380771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-61ad166a-75a5-4141-9ca5-0ade6a4ce5f9 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:44.381720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd23623-d1b9-43cc-ac00-0242903a7ad6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:44.382147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-61ad166a-75a5-4141-9ca5-0ade6a4ce5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:44.470693 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 171/348] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:44.472780 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad166a-75a5-4141-9ca5-0ade6a4ce5f9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:44.527696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd23623-d1b9-43cc-ac00-0242903a7ad6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:44.565656 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:44.585398 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd49d1-7059-4966-ad0a-2facbbe5a122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:44.586219 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd49d1-7059-4966-ad0a-2facbbe5a122 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:44.586611 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd49d1-7059-4966-ad0a-2facbbe5a122 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:44.586995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd49d1-7059-4966-ad0a-2facbbe5a122 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:44.587780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd49d1-7059-4966-ad0a-2facbbe5a122 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:44.588693 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd49d1-7059-4966-ad0a-2facbbe5a122 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:44.590236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd49d1-7059-4966-ad0a-2facbbe5a122 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:44.599088 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 178/349] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:45:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:44.599952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd49d1-7059-4966-ad0a-2facbbe5a122 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:44.601740 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:44.618531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad166a-75a5-4141-9ca5-0ade6a4ce5f9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:44.668116 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138fd050-2978-4bb4-99ff-935b4ee733c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:44.668791 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fd050-2978-4bb4-99ff-935b4ee733c2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:44.669026 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fd050-2978-4bb4-99ff-935b4ee733c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:44.669301 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fd050-2978-4bb4-99ff-935b4ee733c2 None admin] PATH_INFO: `/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:44.679548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138fd050-2978-4bb4-99ff-935b4ee733c2 None admin] RBAC: Authorizing `identity:get_group(group_id=bec1848965a34e0bb1bdb91aed87f8f5)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:44.681013 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138fd050-2978-4bb4-99ff-935b4ee733c2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:44.683085 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 172/350] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 14 11:45:44 2022] GET /identity/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:44.683940 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138fd050-2978-4bb4-99ff-935b4ee733c2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:44.728696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd49d1-7059-4966-ad0a-2facbbe5a122 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:44.766582 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a7039-f716-49f7-9b0c-4444d8ac22d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:44.767606 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a7039-f716-49f7-9b0c-4444d8ac22d2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:44.768160 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a7039-f716-49f7-9b0c-4444d8ac22d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:44.768691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a7039-f716-49f7-9b0c-4444d8ac22d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:44.769590 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5a7039-f716-49f7-9b0c-4444d8ac22d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:44.770128 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5a7039-f716-49f7-9b0c-4444d8ac22d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:44.771712 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5a7039-f716-49f7-9b0c-4444d8ac22d2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:44.782442 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 179/351] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:44.783623 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a7039-f716-49f7-9b0c-4444d8ac22d2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:44.798452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138fd050-2978-4bb4-99ff-935b4ee733c2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:44.833640 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:44.841913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8fa02-9b43-4f76-997e-dc6b3d0a333c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:44.841913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8fa02-9b43-4f76-997e-dc6b3d0a333c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:44.841913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8fa02-9b43-4f76-997e-dc6b3d0a333c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:44.841913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8fa02-9b43-4f76-997e-dc6b3d0a333c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:44.868645 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:44.895068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a7039-f716-49f7-9b0c-4444d8ac22d2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:44.916482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8fa02-9b43-4f76-997e-dc6b3d0a333c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:44.918027 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8fa02-9b43-4f76-997e-dc6b3d0a333c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:44.936281 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517707e-c238-4e57-977f-8ca256061369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:44.936999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517707e-c238-4e57-977f-8ca256061369 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:44.937259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517707e-c238-4e57-977f-8ca256061369 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:44.937539 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517707e-c238-4e57-977f-8ca256061369 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:44.938330 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5517707e-c238-4e57-977f-8ca256061369 None admin] RBAC: Adding query filter params (group.id=bec1848965a34e0bb1bdb91aed87f8f5, role.id=d96ee0e293244ae6a04e8ce652aea68e, scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:44.938644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5517707e-c238-4e57-977f-8ca256061369 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:44.940288 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5517707e-c238-4e57-977f-8ca256061369 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:44.946943 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 180/352] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 14 11:45:44 2022] GET /identity/v3/role_assignments?group.id=bec1848965a34e0bb1bdb91aed87f8f5&scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d&role.id=d96ee0e293244ae6a04e8ce652aea68e => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:44.948279 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599a87b-3abf-42c0-9276-d50e93a2c3e9 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:44.948537 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599a87b-3abf-42c0-9276-d50e93a2c3e9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:44.948942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599a87b-3abf-42c0-9276-d50e93a2c3e9 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:44.950475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 181/353] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:44.951952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599a87b-3abf-42c0-9276-d50e93a2c3e9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:45.046282 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 173/354] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:45.048522 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8fa02-9b43-4f76-997e-dc6b3d0a333c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:45.088985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599a87b-3abf-42c0-9276-d50e93a2c3e9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:45.098375 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:45.130111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:45.138253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948f3c6-d8ad-4c23-8b33-92ed06bc6d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:45.139803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948f3c6-d8ad-4c23-8b33-92ed06bc6d91 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:45.140336 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948f3c6-d8ad-4c23-8b33-92ed06bc6d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:45.140621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948f3c6-d8ad-4c23-8b33-92ed06bc6d91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:45.185577 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8fa02-9b43-4f76-997e-dc6b3d0a333c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:45.233447 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1124d7e-f5e4-46d1-9c6f-30cf830e0b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:45.234516 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1124d7e-f5e4-46d1-9c6f-30cf830e0b00 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:45.234803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1124d7e-f5e4-46d1-9c6f-30cf830e0b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:45.235080 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1124d7e-f5e4-46d1-9c6f-30cf830e0b00 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:45.240200 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948f3c6-d8ad-4c23-8b33-92ed06bc6d91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:45.241670 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948f3c6-d8ad-4c23-8b33-92ed06bc6d91 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:45.246394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1124d7e-f5e4-46d1-9c6f-30cf830e0b00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:45.252580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-e1124d7e-f5e4-46d1-9c6f-30cf830e0b00 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 11:45:45.252580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:45.252580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:45.252580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:45.252580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:45.252580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:45.252580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:45.252580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:45.252580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:45.252580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:45.252580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:45.252580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:45.252580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:45.252580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:45.252580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:45.252580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:45.253487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:45.253487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:45.253487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:45.253487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:45.253487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:45.253487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:45.253487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:45.253487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:45.253487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:45.253487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:45.253487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:45.253487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:45.253487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:45.253487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:45.253487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:45.253487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:45.254371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:45.254371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:45.254371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:45.254371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:45.254371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:45.254371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:45.254371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:45.254371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:45.254371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:45.254371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:45.254371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 11:45:45.254371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:45.254371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 174/355] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:45.255040 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb98ee-b7d5-4121-bf18-ecc3d938b249 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:45.255154 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb98ee-b7d5-4121-bf18-ecc3d938b249 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:45.255428 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb98ee-b7d5-4121-bf18-ecc3d938b249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:45.275701 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2feb98ee-b7d5-4121-bf18-ecc3d938b249 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:45.276913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2feb98ee-b7d5-4121-bf18-ecc3d938b249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:45.365430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 175/356] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:45.366226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb98ee-b7d5-4121-bf18-ecc3d938b249 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:45.371119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 182/357] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:45.374199 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd95a44-1442-4f64-b715-7ec9df45c5ed None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:45.374682 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd95a44-1442-4f64-b715-7ec9df45c5ed None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:45.375140 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd95a44-1442-4f64-b715-7ec9df45c5ed None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:45.376643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 183/358] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:45.377615 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd95a44-1442-4f64-b715-7ec9df45c5ed None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:45.389626 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:45.427861 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:45.492032 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb98ee-b7d5-4121-bf18-ecc3d938b249 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:45.517265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd95a44-1442-4f64-b715-7ec9df45c5ed None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:45.538240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2e3d3-ad46-4002-ac02-5133368bdd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:45.539234 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2e3d3-ad46-4002-ac02-5133368bdd3f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:45.539696 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2e3d3-ad46-4002-ac02-5133368bdd3f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:45.540154 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2e3d3-ad46-4002-ac02-5133368bdd3f None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:45.541466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2e3d3-ad46-4002-ac02-5133368bdd3f None admin] RBAC: Authorizing `identity:get_project(project_id=19b75bd757df4c8cac09dd67b1f0ca8d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:45.543258 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2e3d3-ad46-4002-ac02-5133368bdd3f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:45.545593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 176/359] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:45 2022] GET /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:45.546615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2e3d3-ad46-4002-ac02-5133368bdd3f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:45.569517 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42220698-b831-4929-9954-ea23803ae187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:45.569748 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42220698-b831-4929-9954-ea23803ae187 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:45.570023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42220698-b831-4929-9954-ea23803ae187 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:45.570323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42220698-b831-4929-9954-ea23803ae187 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:45.571009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42220698-b831-4929-9954-ea23803ae187 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:45.571319 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42220698-b831-4929-9954-ea23803ae187 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:45.573085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42220698-b831-4929-9954-ea23803ae187 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:45.584249 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 184/360] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:45.586314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5ffb2-b83d-421d-b467-d470a3f3d793 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:45.586612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5ffb2-b83d-421d-b467-d470a3f3d793 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:45.586896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5ffb2-b83d-421d-b467-d470a3f3d793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:45.613462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-5fc5ffb2-b83d-421d-b467-d470a3f3d793 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:45.614720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5fc5ffb2-b83d-421d-b467-d470a3f3d793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:45.668848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2e3d3-ad46-4002-ac02-5133368bdd3f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:45.697245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:45.703312 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 185/361] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:45.704269 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5ffb2-b83d-421d-b467-d470a3f3d793 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:45.708887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:45.714900 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b02700-0abf-44af-8238-c307ff64b4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:45.715552 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b02700-0abf-44af-8238-c307ff64b4af None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:45.715837 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b02700-0abf-44af-8238-c307ff64b4af None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:45.716137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b02700-0abf-44af-8238-c307ff64b4af None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:45.717920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b02700-0abf-44af-8238-c307ff64b4af None admin] RBAC: Authorizing `identity:get_project(project_id=2164ed8287f243bc9a4a5347d4dc0b1d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:45.719746 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b02700-0abf-44af-8238-c307ff64b4af None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:45.722192 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 177/362] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:45 2022] GET /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:45.723311 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b02700-0abf-44af-8238-c307ff64b4af None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:45.821499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5ffb2-b83d-421d-b467-d470a3f3d793 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:45.871899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28eb70e-cf33-47f3-bdd4-4e226afb3188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:45.872944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28eb70e-cf33-47f3-bdd4-4e226afb3188 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:45.873471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28eb70e-cf33-47f3-bdd4-4e226afb3188 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:45.874030 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28eb70e-cf33-47f3-bdd4-4e226afb3188 None admin] PATH_INFO: `/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:45.875420 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28eb70e-cf33-47f3-bdd4-4e226afb3188 None admin] RBAC: Authorizing `identity:get_group(group_id=bec1848965a34e0bb1bdb91aed87f8f5)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:45.877145 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28eb70e-cf33-47f3-bdd4-4e226afb3188 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:45.879269 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 186/363] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 14 11:45:45 2022] GET /identity/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:45.880007 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b02700-0abf-44af-8238-c307ff64b4af None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:45.880480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28eb70e-cf33-47f3-bdd4-4e226afb3188 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:45.919332 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6188e9-cf17-411b-b6a2-bd0442b0ad62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:45.919879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6188e9-cf17-411b-b6a2-bd0442b0ad62 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:45.920695 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6188e9-cf17-411b-b6a2-bd0442b0ad62 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:45.920981 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6188e9-cf17-411b-b6a2-bd0442b0ad62 None admin] PATH_INFO: `/v3/users/09a76415895543358b3a19da03c199c4` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:45.922702 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6188e9-cf17-411b-b6a2-bd0442b0ad62 None admin] RBAC: Authorizing `identity:get_user(user_id=09a76415895543358b3a19da03c199c4)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:45.924195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6188e9-cf17-411b-b6a2-bd0442b0ad62 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:45.926070 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 178/364] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:45:45 2022] GET /identity/v3/users/09a76415895543358b3a19da03c199c4 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:45.926854 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6188e9-cf17-411b-b6a2-bd0442b0ad62 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:45.955098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:45.987833 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:45.994004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28eb70e-cf33-47f3-bdd4-4e226afb3188 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:46.036684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7771d0a-317f-4141-81bb-87b0a9ee0528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:46.037627 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7771d0a-317f-4141-81bb-87b0a9ee0528 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:46.038096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7771d0a-317f-4141-81bb-87b0a9ee0528 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:46.038516 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7771d0a-317f-4141-81bb-87b0a9ee0528 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:46.047548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6188e9-cf17-411b-b6a2-bd0442b0ad62 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:46.049770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7771d0a-317f-4141-81bb-87b0a9ee0528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:46.055138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-e7771d0a-317f-4141-81bb-87b0a9ee0528 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:46.055138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:46.055138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:46.055138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:46.055138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:46.055138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:46.055138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:46.055138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:46.055138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:46.055138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:46.055138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:46.055138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:46.055138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:46.055138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:46.055138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:46.055138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:46.056043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:46.056043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:46.056043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:46.056043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:46.056043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:46.056043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:46.056043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:46.056043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:46.056043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:46.056043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:46.056043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:46.056043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:46.056043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:46.056043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:46.056043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:46.056043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:46.057245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:46.057245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:46.057245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:46.057245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:46.057245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:46.057245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:46.057245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:46.057245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:46.057245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:46.057245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:46.057245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:46.057245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:46.057245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 187/365] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:45:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:46.057245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7771d0a-317f-4141-81bb-87b0a9ee0528 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:46.101873 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5261c-24b5-41ec-8b24-71d841c7ec79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:46.102695 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5261c-24b5-41ec-8b24-71d841c7ec79 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:46.103275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5261c-24b5-41ec-8b24-71d841c7ec79 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:46.103563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5261c-24b5-41ec-8b24-71d841c7ec79 None admin] PATH_INFO: `/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:46.104851 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5261c-24b5-41ec-8b24-71d841c7ec79 None admin] RBAC: Authorizing `identity:get_group(group_id=bec1848965a34e0bb1bdb91aed87f8f5)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:46.106329 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5261c-24b5-41ec-8b24-71d841c7ec79 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:46.108262 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 179/366] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 14 11:45:45 2022] GET /identity/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:46.109037 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5261c-24b5-41ec-8b24-71d841c7ec79 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:46.191550 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7771d0a-317f-4141-81bb-87b0a9ee0528 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:46.230291 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:46.234746 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5261c-24b5-41ec-8b24-71d841c7ec79 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:46.242180 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1cc82-f64f-428b-adaa-d223f3159dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:46.242797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1cc82-f64f-428b-adaa-d223f3159dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:46.246719 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1cc82-f64f-428b-adaa-d223f3159dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:46.246719 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1cc82-f64f-428b-adaa-d223f3159dc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:46.246719 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1cc82-f64f-428b-adaa-d223f3159dc1 None admin] RBAC: Adding query filter params (group.id=bec1848965a34e0bb1bdb91aed87f8f5, role.id=1ab8c505dc344769abc347f66e529527, scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:46.246719 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1cc82-f64f-428b-adaa-d223f3159dc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:46.246719 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1cc82-f64f-428b-adaa-d223f3159dc1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:46.261901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 188/367] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 14 11:45:46 2022] GET /identity/v3/role_assignments?group.id=bec1848965a34e0bb1bdb91aed87f8f5&scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d&role.id=1ab8c505dc344769abc347f66e529527 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:46.261901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1cc82-f64f-428b-adaa-d223f3159dc1 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:46.263026 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:46.289184 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c875714-80eb-4a3c-af45-e567d822e448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:46.289810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c875714-80eb-4a3c-af45-e567d822e448 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:46.291982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c875714-80eb-4a3c-af45-e567d822e448 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:46.291982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c875714-80eb-4a3c-af45-e567d822e448 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:46.291982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c875714-80eb-4a3c-af45-e567d822e448 None admin] RBAC: Adding query filter params (role.id=f792865844594321a8a67b38335cdccb, user.id=09a76415895543358b3a19da03c199c4, scope.system=all) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:46.292322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c875714-80eb-4a3c-af45-e567d822e448 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:46.294077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c875714-80eb-4a3c-af45-e567d822e448 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:46.318030 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 180/368] 10.222.0.62 () {56 vars in 1308 bytes} [Wed Sep 14 11:45:46 2022] GET /identity/v3/role_assignments?user.id=09a76415895543358b3a19da03c199c4&scope.system=all&role.id=f792865844594321a8a67b38335cdccb => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:46.319176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c875714-80eb-4a3c-af45-e567d822e448 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:46.445302 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c875714-80eb-4a3c-af45-e567d822e448 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:46.489080 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63994f7-1171-40b8-887f-14efedfa4cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:46.489715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63994f7-1171-40b8-887f-14efedfa4cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:46.490031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63994f7-1171-40b8-887f-14efedfa4cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:46.490305 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63994f7-1171-40b8-887f-14efedfa4cc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:46.490985 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63994f7-1171-40b8-887f-14efedfa4cc0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:46.491293 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63994f7-1171-40b8-887f-14efedfa4cc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:46.492800 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63994f7-1171-40b8-887f-14efedfa4cc0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:46.502817 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 181/369] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:45:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:46.503707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63994f7-1171-40b8-887f-14efedfa4cc0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:46.508905 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1cc82-f64f-428b-adaa-d223f3159dc1 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:46.547833 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:46.562186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384f3e2-7034-4135-8a69-394cb9ff928d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:46.563216 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384f3e2-7034-4135-8a69-394cb9ff928d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:46.563703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384f3e2-7034-4135-8a69-394cb9ff928d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:46.564208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384f3e2-7034-4135-8a69-394cb9ff928d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:46.585088 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384f3e2-7034-4135-8a69-394cb9ff928d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:46.591052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-8384f3e2-7034-4135-8a69-394cb9ff928d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:45:46.591052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:46.591052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:46.591052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:46.591052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:46.591052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:46.591052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:46.591052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:46.591052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:46.591052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:46.591052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:46.591052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:46.591052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:46.591052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:46.591052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:46.591052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:46.592525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:46.592525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:46.592525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:46.592525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:46.592525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:46.592525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:46.592525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:46.592525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:46.592525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:46.592525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:46.592525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:46.592525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:46.592525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:46.592525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:46.592525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:46.592525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:46.593315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:46.593315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:46.593315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:46.593315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:46.593315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:46.593315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:46.593315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:46.593315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:46.593315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:46.593315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:46.593315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:45:46.593315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:46.595390 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 189/370] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:45:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:46.596396 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384f3e2-7034-4135-8a69-394cb9ff928d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:46.642183 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63994f7-1171-40b8-887f-14efedfa4cc0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:46.667017 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:46.692927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b7cca-d94e-4e0d-a1ac-0cc90da7f870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:46.693618 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b7cca-d94e-4e0d-a1ac-0cc90da7f870 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:46.693707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b7cca-d94e-4e0d-a1ac-0cc90da7f870 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:46.693981 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b7cca-d94e-4e0d-a1ac-0cc90da7f870 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:46.694723 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b7cca-d94e-4e0d-a1ac-0cc90da7f870 None admin] RBAC: Adding query filter params (group.id=bec1848965a34e0bb1bdb91aed87f8f5, role.id=d96ee0e293244ae6a04e8ce652aea68e, scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:46.695044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b7cca-d94e-4e0d-a1ac-0cc90da7f870 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:46.697558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b7cca-d94e-4e0d-a1ac-0cc90da7f870 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:46.705023 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 182/371] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 14 11:45:46 2022] GET /identity/v3/role_assignments?group.id=bec1848965a34e0bb1bdb91aed87f8f5&scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d&role.id=d96ee0e293244ae6a04e8ce652aea68e => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:46.705952 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b7cca-d94e-4e0d-a1ac-0cc90da7f870 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:46.741907 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384f3e2-7034-4135-8a69-394cb9ff928d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:46.800713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8529532-b067-497e-947d-4c2d8f8d3f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:46.801214 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8529532-b067-497e-947d-4c2d8f8d3f8b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:46.801502 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8529532-b067-497e-947d-4c2d8f8d3f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:46.801780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8529532-b067-497e-947d-4c2d8f8d3f8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:46.846495 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b7cca-d94e-4e0d-a1ac-0cc90da7f870 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:46.848284 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:46.880900 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8529532-b067-497e-947d-4c2d8f8d3f8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:46.882435 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8529532-b067-497e-947d-4c2d8f8d3f8b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:46.888216 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc53fbf-05c3-4a75-97de-10bd40714293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:46.889363 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc53fbf-05c3-4a75-97de-10bd40714293 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:46.889640 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc53fbf-05c3-4a75-97de-10bd40714293 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:46.889948 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc53fbf-05c3-4a75-97de-10bd40714293 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:46.890680 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc53fbf-05c3-4a75-97de-10bd40714293 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:46.890992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc53fbf-05c3-4a75-97de-10bd40714293 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:46.892577 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc53fbf-05c3-4a75-97de-10bd40714293 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:46.903455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 183/372] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:46.927443 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc53fbf-05c3-4a75-97de-10bd40714293 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:46.943772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:47.004459 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 190/373] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:47.009635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8529532-b067-497e-947d-4c2d8f8d3f8b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:47.120211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc53fbf-05c3-4a75-97de-10bd40714293 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:47.139676 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8529532-b067-497e-947d-4c2d8f8d3f8b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:47.177160 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1243d9-b22c-4271-829a-6eb00ed2c8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:47.177779 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1243d9-b22c-4271-829a-6eb00ed2c8cd None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:47.178158 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1243d9-b22c-4271-829a-6eb00ed2c8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:47.178436 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1243d9-b22c-4271-829a-6eb00ed2c8cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:47.202142 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62977fad-b665-495f-906c-2a6d14c4bc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:47.202783 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62977fad-b665-495f-906c-2a6d14c4bc3c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:47.203062 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62977fad-b665-495f-906c-2a6d14c4bc3c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:47.203336 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62977fad-b665-495f-906c-2a6d14c4bc3c None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:47.204597 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62977fad-b665-495f-906c-2a6d14c4bc3c None admin] RBAC: Authorizing `identity:get_project(project_id=2164ed8287f243bc9a4a5347d4dc0b1d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:47.207715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62977fad-b665-495f-906c-2a6d14c4bc3c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:47.210662 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 191/374] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:47 2022] GET /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:47.214737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62977fad-b665-495f-906c-2a6d14c4bc3c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:47.241532 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:47.259913 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:47.315526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1243d9-b22c-4271-829a-6eb00ed2c8cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:47.316996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1243d9-b22c-4271-829a-6eb00ed2c8cd None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:47.411841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62977fad-b665-495f-906c-2a6d14c4bc3c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:47.490469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4192ea9-6035-4291-b084-2f479cc755b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:47.491158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4192ea9-6035-4291-b084-2f479cc755b6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:47.491441 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4192ea9-6035-4291-b084-2f479cc755b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:47.491715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4192ea9-6035-4291-b084-2f479cc755b6 None admin] PATH_INFO: `/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:47.492999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4192ea9-6035-4291-b084-2f479cc755b6 None admin] RBAC: Authorizing `identity:get_group(group_id=bec1848965a34e0bb1bdb91aed87f8f5)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:47.494537 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4192ea9-6035-4291-b084-2f479cc755b6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:47.496187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 192/375] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 14 11:45:47 2022] GET /identity/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5 => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:47.503672 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4192ea9-6035-4291-b084-2f479cc755b6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:47.522450 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 184/376] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 11:45:47.529220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1243d9-b22c-4271-829a-6eb00ed2c8cd None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:47.616658 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:47.649044 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:47.670721 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4192ea9-6035-4291-b084-2f479cc755b6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:47.671577 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1243d9-b22c-4271-829a-6eb00ed2c8cd None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:47.714528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6856ed3-67c3-46fa-a382-2f602baa72a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:47.715194 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6856ed3-67c3-46fa-a382-2f602baa72a1 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:47.715442 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6856ed3-67c3-46fa-a382-2f602baa72a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:47.715714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6856ed3-67c3-46fa-a382-2f602baa72a1 None admin] PATH_INFO: `/v3/projects/a95d34dd28804fc3b45b9488a19047b4` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:47.716539 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c184af9-b351-40ae-aec7-51180222cfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:47.717135 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6856ed3-67c3-46fa-a382-2f602baa72a1 None admin] RBAC: Authorizing `identity:get_project(project_id=a95d34dd28804fc3b45b9488a19047b4)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:47.717509 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c184af9-b351-40ae-aec7-51180222cfb6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:47.717967 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c184af9-b351-40ae-aec7-51180222cfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:47.718388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c184af9-b351-40ae-aec7-51180222cfb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:47.718599 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6856ed3-67c3-46fa-a382-2f602baa72a1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:47.719428 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c184af9-b351-40ae-aec7-51180222cfb6 None admin] RBAC: Adding query filter params (group.id=bec1848965a34e0bb1bdb91aed87f8f5, role.id=1ab8c505dc344769abc347f66e529527, scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:47.720976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c184af9-b351-40ae-aec7-51180222cfb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:47.725153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 185/377] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:47 2022] GET /identity/v3/projects/a95d34dd28804fc3b45b9488a19047b4 => generated 344 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:47.725153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb0e5d-9af7-465d-8c7b-cd94f678b210 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:47.725153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb0e5d-9af7-465d-8c7b-cd94f678b210 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:47.725153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb0e5d-9af7-465d-8c7b-cd94f678b210 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:47.725153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 186/378] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:47.726450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c184af9-b351-40ae-aec7-51180222cfb6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:47.730538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5adc8-0fbc-4de6-8efb-82f41d6b543e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:47.730902 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5adc8-0fbc-4de6-8efb-82f41d6b543e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:47.731179 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5adc8-0fbc-4de6-8efb-82f41d6b543e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:47.734512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 193/379] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 14 11:45:47 2022] GET /identity/v3/role_assignments?group.id=bec1848965a34e0bb1bdb91aed87f8f5&scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d&role.id=1ab8c505dc344769abc347f66e529527 => generated 260 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:47.735763 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c184af9-b351-40ae-aec7-51180222cfb6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:47.753779 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-fac5adc8-0fbc-4de6-8efb-82f41d6b543e None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:47.755203 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fac5adc8-0fbc-4de6-8efb-82f41d6b543e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:47.835847 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 187/380] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:47.852826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c184af9-b351-40ae-aec7-51180222cfb6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:47.893196 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd967f-b881-4765-940d-b3467fc34f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:47.894573 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd967f-b881-4765-940d-b3467fc34f06 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:47.895008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd967f-b881-4765-940d-b3467fc34f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:47.895465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd967f-b881-4765-940d-b3467fc34f06 None admin] PATH_INFO: `/v3/groups/26009220b25145a1bf4b0da2b68e5b51` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:47.906982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd967f-b881-4765-940d-b3467fc34f06 None admin] RBAC: Authorizing `identity:get_group(group_id=26009220b25145a1bf4b0da2b68e5b51)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:47.908519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd967f-b881-4765-940d-b3467fc34f06 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:47.910286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 194/381] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 14 11:45:47 2022] GET /identity/v3/groups/26009220b25145a1bf4b0da2b68e5b51 => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:47.916427 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5adc8-0fbc-4de6-8efb-82f41d6b543e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:47.995543 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:48.042199 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5adc8-0fbc-4de6-8efb-82f41d6b543e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:48.086094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250fec9-cac7-4dbb-b922-b0986445ea68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:48.086648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250fec9-cac7-4dbb-b922-b0986445ea68 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:48.086931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250fec9-cac7-4dbb-b922-b0986445ea68 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:48.087208 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250fec9-cac7-4dbb-b922-b0986445ea68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:48.087956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250fec9-cac7-4dbb-b922-b0986445ea68 None admin] RBAC: Adding query filter params (group.id=26009220b25145a1bf4b0da2b68e5b51, role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.project.id=a95d34dd28804fc3b45b9488a19047b4) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:48.088266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250fec9-cac7-4dbb-b922-b0986445ea68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:48.092237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250fec9-cac7-4dbb-b922-b0986445ea68 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:48.098548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 188/382] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 14 11:45:47 2022] GET /identity/v3/role_assignments?group.id=26009220b25145a1bf4b0da2b68e5b51&scope.project.id=a95d34dd28804fc3b45b9488a19047b4&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:48.236351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd967f-b881-4765-940d-b3467fc34f06 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:48.259064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:48.390795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd967f-b881-4765-940d-b3467fc34f06 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:48.439136 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7cede-cc4f-4dd1-bb67-3def720e7628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:48.439791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7cede-cc4f-4dd1-bb67-3def720e7628 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:48.440076 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7cede-cc4f-4dd1-bb67-3def720e7628 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:48.440352 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7cede-cc4f-4dd1-bb67-3def720e7628 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:48.464007 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7cede-cc4f-4dd1-bb67-3def720e7628 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:48.470087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-c0b7cede-cc4f-4dd1-bb67-3def720e7628 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:48.470087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:48.470087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:48.470087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:48.470087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:48.470087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:48.470087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:48.470087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:48.470087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:48.470087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:48.470087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:48.470087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:48.470087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:48.470087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:48.470087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:48.470087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:48.470978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:48.470978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:48.470978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:48.470978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:48.470978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:48.470978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:48.470978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:48.470978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:48.470978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:48.470978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:48.470978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:48.470978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:48.470978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:48.470978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:48.470978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:48.470978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:48.471798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:48.471798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:48.471798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:48.471798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:48.471798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:48.471798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:48.471798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:48.471798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:48.471798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:48.471798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:48.471798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:48.471798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:48.471798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 195/383] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:48.479080 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250fec9-cac7-4dbb-b922-b0986445ea68 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:48.611925 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250fec9-cac7-4dbb-b922-b0986445ea68 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:48.647524 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:48.658799 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f90eb-7b41-460e-9b5f-fbdef4d5092c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:48.659419 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f90eb-7b41-460e-9b5f-fbdef4d5092c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:48.659760 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f90eb-7b41-460e-9b5f-fbdef4d5092c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:48.660044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f90eb-7b41-460e-9b5f-fbdef4d5092c None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:48.660717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f90eb-7b41-460e-9b5f-fbdef4d5092c None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:48.661024 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f90eb-7b41-460e-9b5f-fbdef4d5092c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:48.662566 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f90eb-7b41-460e-9b5f-fbdef4d5092c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:48.673308 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 189/384] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:48.679316 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7cede-cc4f-4dd1-bb67-3def720e7628 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:48.797001 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:48.833547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7cede-cc4f-4dd1-bb67-3def720e7628 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:48.888006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad8bc0-08ba-4890-84ff-6bfdc3b3a49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:48.888768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad8bc0-08ba-4890-84ff-6bfdc3b3a49e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:48.889483 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad8bc0-08ba-4890-84ff-6bfdc3b3a49e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:48.889760 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad8bc0-08ba-4890-84ff-6bfdc3b3a49e None admin] PATH_INFO: `/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:48.890934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad8bc0-08ba-4890-84ff-6bfdc3b3a49e None admin] RBAC: Authorizing `identity:get_group(group_id=bec1848965a34e0bb1bdb91aed87f8f5)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:48.892433 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad8bc0-08ba-4890-84ff-6bfdc3b3a49e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:48.894032 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 196/385] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 14 11:45:48 2022] GET /identity/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:48.900306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f90eb-7b41-460e-9b5f-fbdef4d5092c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:48.961361 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd9725-a8ca-4a4f-89cf-384ec764a98c None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:48.961679 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd9725-a8ca-4a4f-89cf-384ec764a98c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:48.962073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd9725-a8ca-4a4f-89cf-384ec764a98c None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:48.963422 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 197/386] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:48.968587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a4011-a3ea-4dd1-a401-5349f3cdccd8 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:48.968874 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a4011-a3ea-4dd1-a401-5349f3cdccd8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:48.969151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a4011-a3ea-4dd1-a401-5349f3cdccd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:49.003064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-6c1a4011-a3ea-4dd1-a401-5349f3cdccd8 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:49.004423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6c1a4011-a3ea-4dd1-a401-5349f3cdccd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:49.028932 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f90eb-7b41-460e-9b5f-fbdef4d5092c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:49.073804 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfebe65-2ee3-4aa9-a5c4-d85ab1ff9240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:49.074790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfebe65-2ee3-4aa9-a5c4-d85ab1ff9240 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:49.075255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfebe65-2ee3-4aa9-a5c4-d85ab1ff9240 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:49.075711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfebe65-2ee3-4aa9-a5c4-d85ab1ff9240 None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:49.077689 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfebe65-2ee3-4aa9-a5c4-d85ab1ff9240 None admin] RBAC: Authorizing `identity:get_project(project_id=19b75bd757df4c8cac09dd67b1f0ca8d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:49.079478 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfebe65-2ee3-4aa9-a5c4-d85ab1ff9240 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:49.082751 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 190/387] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:48 2022] GET /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:49.090126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfebe65-2ee3-4aa9-a5c4-d85ab1ff9240 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:49.158316 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 198/388] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:45:49.194899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:49.200269 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfebe65-2ee3-4aa9-a5c4-d85ab1ff9240 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:49.236892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad235c21-c08d-4cb6-a546-7e5299739327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:49.237915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad235c21-c08d-4cb6-a546-7e5299739327 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:49.238400 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad235c21-c08d-4cb6-a546-7e5299739327 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:49.238964 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad235c21-c08d-4cb6-a546-7e5299739327 None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d/groups/bec1848965a34e0bb1bdb91aed87f8f5/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:49.241260 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad235c21-c08d-4cb6-a546-7e5299739327 None admin] RBAC: Authorizing `identity:create_grant(project_id=19b75bd757df4c8cac09dd67b1f0ca8d, group_id=bec1848965a34e0bb1bdb91aed87f8f5, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:49.243054 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad235c21-c08d-4cb6-a546-7e5299739327 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:49.330524 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 191/389] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Sep 14 11:45:49 2022] PUT /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d/groups/bec1848965a34e0bb1bdb91aed87f8f5/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:45:49.487680 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a4011-a3ea-4dd1-a401-5349f3cdccd8 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:49.579245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:49.609440 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a4011-a3ea-4dd1-a401-5349f3cdccd8 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:49.651251 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54be46c-056d-45e4-9675-33ecaa40f86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:49.651910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54be46c-056d-45e4-9675-33ecaa40f86b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:49.652194 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54be46c-056d-45e4-9675-33ecaa40f86b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:49.652469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54be46c-056d-45e4-9675-33ecaa40f86b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:49.665594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54be46c-056d-45e4-9675-33ecaa40f86b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:49.670780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-d54be46c-056d-45e4-9675-33ecaa40f86b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:49.670780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:49.670780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:49.670780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:49.670780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:49.670780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:49.670780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:49.670780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:49.670780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:49.670780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:49.670780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:49.670780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:49.670780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:49.670780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:49.670780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:49.670780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:49.671767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:49.671767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:49.671767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:49.671767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:49.671767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:49.671767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:49.671767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:49.671767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:49.671767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:49.671767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:49.671767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:49.671767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:49.671767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:49.671767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:49.671767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:49.671767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:49.672567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:49.672567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:49.672567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:49.672567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:49.672567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:49.672567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:49.672567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:49.672567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:49.672567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:49.672567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:49.672567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:49.672567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:49.672567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 199/390] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:45:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:49.678722 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad235c21-c08d-4cb6-a546-7e5299739327 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:49.785436 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08b616-44fa-4441-8bc5-2eee9d6af4dc None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:49.785685 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08b616-44fa-4441-8bc5-2eee9d6af4dc None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:49.785957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08b616-44fa-4441-8bc5-2eee9d6af4dc None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:49.787453 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 200/391] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:45:49.792816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a10f22-6c49-4334-a348-35cef474f72f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:49.793357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a10f22-6c49-4334-a348-35cef474f72f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:49.793835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a10f22-6c49-4334-a348-35cef474f72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:49.816282 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-00a10f22-6c49-4334-a348-35cef474f72f None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:49.817893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-00a10f22-6c49-4334-a348-35cef474f72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:49.829192 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad235c21-c08d-4cb6-a546-7e5299739327 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:49.861626 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:49.896466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70680c9-e115-43b8-9113-8807b3aa8231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:49.897091 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70680c9-e115-43b8-9113-8807b3aa8231 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:49.897368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70680c9-e115-43b8-9113-8807b3aa8231 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:49.897631 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70680c9-e115-43b8-9113-8807b3aa8231 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:49.900148 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70680c9-e115-43b8-9113-8807b3aa8231 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:49.900476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70680c9-e115-43b8-9113-8807b3aa8231 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:49.901884 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70680c9-e115-43b8-9113-8807b3aa8231 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:49.917170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 192/392] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:45:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:49.922155 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70680c9-e115-43b8-9113-8807b3aa8231 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:49.950874 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 201/393] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:45:50.053948 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70680c9-e115-43b8-9113-8807b3aa8231 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:50.088596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31adcfd6-799b-438d-8399-969a65807246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:50.089238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adcfd6-799b-438d-8399-969a65807246 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:50.089520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adcfd6-799b-438d-8399-969a65807246 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:50.089794 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adcfd6-799b-438d-8399-969a65807246 None admin] PATH_INFO: `/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:50.090942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31adcfd6-799b-438d-8399-969a65807246 None admin] RBAC: Authorizing `identity:get_group(group_id=bec1848965a34e0bb1bdb91aed87f8f5)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:50.092480 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31adcfd6-799b-438d-8399-969a65807246 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:50.094420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 193/394] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 14 11:45:49 2022] GET /identity/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:50.100032 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a10f22-6c49-4334-a348-35cef474f72f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:50.170630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:50.221091 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a10f22-6c49-4334-a348-35cef474f72f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:50.260203 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a92dd-a723-4bda-8bce-b5f9317d1181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:50.260899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a92dd-a723-4bda-8bce-b5f9317d1181 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:50.261241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a92dd-a723-4bda-8bce-b5f9317d1181 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:50.261522 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a92dd-a723-4bda-8bce-b5f9317d1181 None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:50.262709 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a92dd-a723-4bda-8bce-b5f9317d1181 None admin] RBAC: Authorizing `identity:get_project(project_id=19b75bd757df4c8cac09dd67b1f0ca8d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:50.264187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a92dd-a723-4bda-8bce-b5f9317d1181 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:50.266116 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 202/395] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:50 2022] GET /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:50.271983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31adcfd6-799b-438d-8399-969a65807246 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:50.330404 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:50.366811 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a92dd-a723-4bda-8bce-b5f9317d1181 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:50.387787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31adcfd6-799b-438d-8399-969a65807246 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:50.430155 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db450d-d43f-4292-82f5-ba6e54540a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:50.430820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db450d-d43f-4292-82f5-ba6e54540a1f None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:50.431148 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db450d-d43f-4292-82f5-ba6e54540a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:50.431425 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db450d-d43f-4292-82f5-ba6e54540a1f None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d/groups/bec1848965a34e0bb1bdb91aed87f8f5/roles/1ab8c505dc344769abc347f66e529527` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:50.433555 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db450d-d43f-4292-82f5-ba6e54540a1f None admin] RBAC: Authorizing `identity:create_grant(project_id=19b75bd757df4c8cac09dd67b1f0ca8d, group_id=bec1848965a34e0bb1bdb91aed87f8f5, role_id=1ab8c505dc344769abc347f66e529527)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:50.435144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db450d-d43f-4292-82f5-ba6e54540a1f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:50.499540 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a92dd-a723-4bda-8bce-b5f9317d1181 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:50.506555 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 194/396] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Sep 14 11:45:50 2022] PUT /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d/groups/bec1848965a34e0bb1bdb91aed87f8f5/roles/1ab8c505dc344769abc347f66e529527 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:45:50.541195 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:50.548656 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e8e2b-4c42-4708-8371-53dad3d6c7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:50.549397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e8e2b-4c42-4708-8371-53dad3d6c7ed None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:50.549674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e8e2b-4c42-4708-8371-53dad3d6c7ed None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:50.550307 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e8e2b-4c42-4708-8371-53dad3d6c7ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:50.563745 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e8e2b-4c42-4708-8371-53dad3d6c7ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:50.569988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-b50e8e2b-4c42-4708-8371-53dad3d6c7ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:50.569988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:50.569988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:50.569988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:50.569988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:50.569988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:50.569988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:50.569988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:50.569988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:50.569988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:50.569988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:50.569988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:50.569988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:50.569988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:50.569988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:50.569988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:50.570946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:50.570946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:50.570946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:50.570946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:50.570946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:50.570946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:50.570946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:50.570946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:50.570946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:50.570946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:50.570946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:50.570946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:50.570946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:50.570946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:50.570946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:50.570946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:50.571805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:50.571805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:50.571805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:50.571805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:50.571805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:50.571805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:50.571805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:50.571805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:50.571805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:50.571805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:50.571805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:50.571805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:50.571805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 203/397] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:50.577531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db450d-d43f-4292-82f5-ba6e54540a1f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:50.735692 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db450d-d43f-4292-82f5-ba6e54540a1f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:50.744209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:50.785424 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f486653-1a4a-4c8c-bfa9-b43736101579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:50.786099 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f486653-1a4a-4c8c-bfa9-b43736101579 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:50.786375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f486653-1a4a-4c8c-bfa9-b43736101579 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:50.786648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f486653-1a4a-4c8c-bfa9-b43736101579 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:50.787321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f486653-1a4a-4c8c-bfa9-b43736101579 None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:50.787627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f486653-1a4a-4c8c-bfa9-b43736101579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:50.790433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f486653-1a4a-4c8c-bfa9-b43736101579 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:50.801833 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 195/398] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:50.810159 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e8e2b-4c42-4708-8371-53dad3d6c7ed None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:50.942377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e8e2b-4c42-4708-8371-53dad3d6c7ed None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:50.984798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac842d-7b96-4112-874c-a678d1550828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:50.985448 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac842d-7b96-4112-874c-a678d1550828 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:50.985765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac842d-7b96-4112-874c-a678d1550828 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:50.986000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac842d-7b96-4112-874c-a678d1550828 None admin] PATH_INFO: `/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:50.990519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac842d-7b96-4112-874c-a678d1550828 None admin] RBAC: Authorizing `identity:get_group(group_id=bec1848965a34e0bb1bdb91aed87f8f5)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:50.992228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac842d-7b96-4112-874c-a678d1550828 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:50.993846 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b7f02-23fb-428f-9f3c-f6328b539c1a None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:50.994311 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 204/399] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 14 11:45:50 2022] GET /identity/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:50.994412 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b7f02-23fb-428f-9f3c-f6328b539c1a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:50.994508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b7f02-23fb-428f-9f3c-f6328b539c1a None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:50.997021 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 196/400] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:45:51.000004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac842d-7b96-4112-874c-a678d1550828 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:51.011617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c5305-2e61-4d13-92c3-84e8c63abfe8 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:51.011929 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c5305-2e61-4d13-92c3-84e8c63abfe8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:51.012186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c5305-2e61-4d13-92c3-84e8c63abfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:51.044413 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3a3c5305-2e61-4d13-92c3-84e8c63abfe8 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:51.045679 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3a3c5305-2e61-4d13-92c3-84e8c63abfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:51.082237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:51.127151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac842d-7b96-4112-874c-a678d1550828 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:51.187793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 197/401] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:51.197656 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b170c3-aab1-4717-b51c-90936f70d050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:51.198326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b170c3-aab1-4717-b51c-90936f70d050 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:51.198607 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b170c3-aab1-4717-b51c-90936f70d050 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:51.198885 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b170c3-aab1-4717-b51c-90936f70d050 None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:51.200100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b170c3-aab1-4717-b51c-90936f70d050 None admin] RBAC: Authorizing `identity:get_project(project_id=2164ed8287f243bc9a4a5347d4dc0b1d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:51.202230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b170c3-aab1-4717-b51c-90936f70d050 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:51.204155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 205/402] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:50 2022] GET /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:51.209461 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c5305-2e61-4d13-92c3-84e8c63abfe8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:51.320568 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:51.324360 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c5305-2e61-4d13-92c3-84e8c63abfe8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:51.359520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fab64-980e-4b59-b6e0-06f9ddd6c455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:51.360182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fab64-980e-4b59-b6e0-06f9ddd6c455 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:51.360457 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fab64-980e-4b59-b6e0-06f9ddd6c455 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:51.360726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fab64-980e-4b59-b6e0-06f9ddd6c455 None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d/groups/bec1848965a34e0bb1bdb91aed87f8f5/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:51.362665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729fab64-980e-4b59-b6e0-06f9ddd6c455 None admin] RBAC: Authorizing `identity:create_grant(project_id=2164ed8287f243bc9a4a5347d4dc0b1d, group_id=bec1848965a34e0bb1bdb91aed87f8f5, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:51.364168 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729fab64-980e-4b59-b6e0-06f9ddd6c455 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:51.410579 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99fc5d-1798-4462-9b76-338e478c523a None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:51.410897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99fc5d-1798-4462-9b76-338e478c523a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:51.411191 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99fc5d-1798-4462-9b76-338e478c523a None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:51.412962 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 206/403] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:51.417605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62ba6d-8887-4d72-8f10-f581fb138c1e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:51.417930 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62ba6d-8887-4d72-8f10-f581fb138c1e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:51.418218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62ba6d-8887-4d72-8f10-f581fb138c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:51.438446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9f62ba6d-8887-4d72-8f10-f581fb138c1e None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:51.439839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9f62ba6d-8887-4d72-8f10-f581fb138c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:51.475632 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 198/404] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Sep 14 11:45:51 2022] PUT /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d/groups/bec1848965a34e0bb1bdb91aed87f8f5/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:45:51.545539 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fab64-980e-4b59-b6e0-06f9ddd6c455 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:51.549008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 207/405] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:45:51.674669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fab64-980e-4b59-b6e0-06f9ddd6c455 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:51.710021 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77080eb0-c676-4618-acc7-35ae2c6a301b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:51.710665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77080eb0-c676-4618-acc7-35ae2c6a301b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:51.710954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77080eb0-c676-4618-acc7-35ae2c6a301b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:51.711229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77080eb0-c676-4618-acc7-35ae2c6a301b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:51.722202 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77080eb0-c676-4618-acc7-35ae2c6a301b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:51.727120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-77080eb0-c676-4618-acc7-35ae2c6a301b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:51.727120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:51.727120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:51.727120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:51.727120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:51.727120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:51.727120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:51.727120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:51.727120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:51.727120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:51.727120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:51.727120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:51.727120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:51.727120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:51.727120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:51.727120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:51.728055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:51.728055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:51.728055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:51.728055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:51.728055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:51.728055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:51.728055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:51.728055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:51.728055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:51.728055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:51.728055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:51.728055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:51.728055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:51.728055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:51.728055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:51.728055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:51.728840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:51.728840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:51.728840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:51.728840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:51.728840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:51.728840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:51.728840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:51.728840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:51.728840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:51.728840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:51.728840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:51.728840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:51.728840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 199/406] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:45:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:45:51.739068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62ba6d-8887-4d72-8f10-f581fb138c1e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:51.807490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:51.874256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62ba6d-8887-4d72-8f10-f581fb138c1e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:51.883201 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77080eb0-c676-4618-acc7-35ae2c6a301b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:51.907926 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:51.938677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca9b59-1fc8-4dce-bef9-e9100ba86302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:51.939319 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca9b59-1fc8-4dce-bef9-e9100ba86302 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:51.939612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca9b59-1fc8-4dce-bef9-e9100ba86302 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:51.939888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca9b59-1fc8-4dce-bef9-e9100ba86302 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:51.940584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca9b59-1fc8-4dce-bef9-e9100ba86302 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:51.940889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca9b59-1fc8-4dce-bef9-e9100ba86302 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:51.942340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca9b59-1fc8-4dce-bef9-e9100ba86302 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:51.955281 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 208/407] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:45:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:51.960465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca9b59-1fc8-4dce-bef9-e9100ba86302 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:52.036782 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77080eb0-c676-4618-acc7-35ae2c6a301b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:52.104298 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a1a72-164f-4f6a-a2dc-5a3d09fb8e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:52.106236 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a1a72-164f-4f6a-a2dc-5a3d09fb8e44 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:52.106532 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a1a72-164f-4f6a-a2dc-5a3d09fb8e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:52.106819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a1a72-164f-4f6a-a2dc-5a3d09fb8e44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:52.119303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a1a72-164f-4f6a-a2dc-5a3d09fb8e44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:52.124339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-932a1a72-164f-4f6a-a2dc-5a3d09fb8e44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:45:52.124339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:52.124339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:52.124339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:52.124339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:52.124339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:52.124339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:52.124339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:52.124339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:52.124339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:52.124339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:52.124339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:52.124339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:52.124339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:52.124339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:52.124339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:52.125240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:52.125240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:52.125240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:52.125240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:52.125240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:52.125240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:52.125240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:52.125240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:52.125240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:52.125240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:52.125240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:52.125240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:52.125240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:52.125240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:52.125240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:52.125240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:52.126102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:52.126102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:52.126102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:52.126102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:52.126102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:52.126102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:52.126102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:52.126102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:52.126102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:52.126102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:52.126102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:45:52.126102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:52.126102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 200/408] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:45:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:52.127261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b1461-ecbb-4bb1-beff-2d4820136e6f None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:52.127261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b1461-ecbb-4bb1-beff-2d4820136e6f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:52.127522 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b1461-ecbb-4bb1-beff-2d4820136e6f None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:52.134213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 201/409] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:52.135223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b1461-ecbb-4bb1-beff-2d4820136e6f None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:52.136081 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca9b59-1fc8-4dce-bef9-e9100ba86302 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:52.155645 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:52.173106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d7914-a524-4c6b-9214-bfd2588942fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:52.173793 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d7914-a524-4c6b-9214-bfd2588942fb None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:52.174103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d7914-a524-4c6b-9214-bfd2588942fb None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:52.174382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d7914-a524-4c6b-9214-bfd2588942fb None admin] PATH_INFO: `/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:52.175754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d7914-a524-4c6b-9214-bfd2588942fb None admin] RBAC: Authorizing `identity:get_group(group_id=bec1848965a34e0bb1bdb91aed87f8f5)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:52.177191 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d7914-a524-4c6b-9214-bfd2588942fb None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:52.179455 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 209/410] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 14 11:45:51 2022] GET /identity/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:52.180614 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657d13d-d5af-4c4a-93cb-6660afeaf1fd None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:52.180993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657d13d-d5af-4c4a-93cb-6660afeaf1fd None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:52.181272 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657d13d-d5af-4c4a-93cb-6660afeaf1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:52.200282 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-6657d13d-d5af-4c4a-93cb-6660afeaf1fd None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:52.201984 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6657d13d-d5af-4c4a-93cb-6660afeaf1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:52.235949 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:52.253797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b1461-ecbb-4bb1-beff-2d4820136e6f None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:52.272521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 210/411] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:52.273741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657d13d-d5af-4c4a-93cb-6660afeaf1fd None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:52.323596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f2201-94ef-4dd6-9bae-0975c69c65a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:52.324485 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f2201-94ef-4dd6-9bae-0975c69c65a0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:52.324874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f2201-94ef-4dd6-9bae-0975c69c65a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:52.325254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f2201-94ef-4dd6-9bae-0975c69c65a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:52.326678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f2201-94ef-4dd6-9bae-0975c69c65a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:52.327088 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f2201-94ef-4dd6-9bae-0975c69c65a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:52.328576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f2201-94ef-4dd6-9bae-0975c69c65a0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:52.353135 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 202/412] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:52.358824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f2201-94ef-4dd6-9bae-0975c69c65a0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:52.379574 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657d13d-d5af-4c4a-93cb-6660afeaf1fd None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:52.398184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:52.417212 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c04886-900e-4d26-b501-e17250c6da1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:52.418236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c04886-900e-4d26-b501-e17250c6da1c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:52.418764 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c04886-900e-4d26-b501-e17250c6da1c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:52.419251 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c04886-900e-4d26-b501-e17250c6da1c None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:52.420616 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c04886-900e-4d26-b501-e17250c6da1c None admin] RBAC: Authorizing `identity:get_project(project_id=2164ed8287f243bc9a4a5347d4dc0b1d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:52.422341 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c04886-900e-4d26-b501-e17250c6da1c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:52.424235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 211/413] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:52 2022] GET /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:52.430505 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c04886-900e-4d26-b501-e17250c6da1c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:52.537230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c04886-900e-4d26-b501-e17250c6da1c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:52.564014 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f2201-94ef-4dd6-9bae-0975c69c65a0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:52.574002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd0810-b24c-42a0-bca5-9ce5b45344c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:52.575083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd0810-b24c-42a0-bca5-9ce5b45344c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:52.575561 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd0810-b24c-42a0-bca5-9ce5b45344c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:52.576649 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd0810-b24c-42a0-bca5-9ce5b45344c4 None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d/groups/bec1848965a34e0bb1bdb91aed87f8f5/roles/1ab8c505dc344769abc347f66e529527` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:52.579143 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd0810-b24c-42a0-bca5-9ce5b45344c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=2164ed8287f243bc9a4a5347d4dc0b1d, group_id=bec1848965a34e0bb1bdb91aed87f8f5, role_id=1ab8c505dc344769abc347f66e529527)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:52.580891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd0810-b24c-42a0-bca5-9ce5b45344c4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:52.606515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4af907-7577-421f-b6fa-3cee1bb07f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:52.607146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4af907-7577-421f-b6fa-3cee1bb07f2f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:52.607453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4af907-7577-421f-b6fa-3cee1bb07f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:52.607729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4af907-7577-421f-b6fa-3cee1bb07f2f None admin] PATH_INFO: `/v3/groups/26009220b25145a1bf4b0da2b68e5b51` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:52.608919 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4af907-7577-421f-b6fa-3cee1bb07f2f None admin] RBAC: Authorizing `identity:get_group(group_id=26009220b25145a1bf4b0da2b68e5b51)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:52.610382 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4af907-7577-421f-b6fa-3cee1bb07f2f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:52.612181 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 203/414] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 14 11:45:52 2022] GET /identity/v3/groups/26009220b25145a1bf4b0da2b68e5b51 => generated 229 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:52.612748 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4af907-7577-421f-b6fa-3cee1bb07f2f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:52.630759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:52.640913 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 212/415] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Sep 14 11:45:52 2022] PUT /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d/groups/bec1848965a34e0bb1bdb91aed87f8f5/roles/1ab8c505dc344769abc347f66e529527 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 11:45:52.642242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd0810-b24c-42a0-bca5-9ce5b45344c4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:52.723816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:52.793979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd0810-b24c-42a0-bca5-9ce5b45344c4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:52.850599 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97129e05-9aba-4f48-9857-c7dd2c917d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:52.851267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97129e05-9aba-4f48-9857-c7dd2c917d17 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:52.851545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97129e05-9aba-4f48-9857-c7dd2c917d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:52.851825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97129e05-9aba-4f48-9857-c7dd2c917d17 None admin] PATH_INFO: `/v3/projects/a95d34dd28804fc3b45b9488a19047b4` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:52.853267 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4af907-7577-421f-b6fa-3cee1bb07f2f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:52.855609 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97129e05-9aba-4f48-9857-c7dd2c917d17 None admin] RBAC: Authorizing `identity:get_project(project_id=a95d34dd28804fc3b45b9488a19047b4)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:52.857124 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97129e05-9aba-4f48-9857-c7dd2c917d17 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:52.866919 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 213/416] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:52 2022] GET /identity/v3/projects/a95d34dd28804fc3b45b9488a19047b4 => generated 344 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:52.870539 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97129e05-9aba-4f48-9857-c7dd2c917d17 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:52.906184 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27fcf5-51e6-4312-8e3b-abcfbb039271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:52.906927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27fcf5-51e6-4312-8e3b-abcfbb039271 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:52.907216 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27fcf5-51e6-4312-8e3b-abcfbb039271 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:52.907589 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27fcf5-51e6-4312-8e3b-abcfbb039271 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:52.919641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27fcf5-51e6-4312-8e3b-abcfbb039271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:52.926188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-be27fcf5-51e6-4312-8e3b-abcfbb039271 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:52.926188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:52.926188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:52.926188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:52.926188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:52.926188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:52.926188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:52.926188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:52.926188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:52.926188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:52.926188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:52.926188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:52.926188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:52.926188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:52.926188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:52.926188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:52.927165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:52.927165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:52.927165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:52.927165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:52.927165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:52.927165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:52.927165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:52.927165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:52.927165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:52.927165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:52.927165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:52.927165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:52.927165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:52.927165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:52.927165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:52.927165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:52.929375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:52.929375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:52.929375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:52.929375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:52.929375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:52.929375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:52.929375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:52.929375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:52.929375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:52.929375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:52.929375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:52.929375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:52.929375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 204/417] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:52.933175 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27fcf5-51e6-4312-8e3b-abcfbb039271 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:52.992094 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97129e05-9aba-4f48-9857-c7dd2c917d17 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:53.011447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:53.018416 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:53.038208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d040a98-9fa3-4bf2-ba19-2dbd804740f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:53.039205 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d040a98-9fa3-4bf2-ba19-2dbd804740f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:53.039205 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d040a98-9fa3-4bf2-ba19-2dbd804740f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:53.039462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d040a98-9fa3-4bf2-ba19-2dbd804740f8 None admin] PATH_INFO: `/v3/projects/a95d34dd28804fc3b45b9488a19047b4/groups/26009220b25145a1bf4b0da2b68e5b51/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:53.041591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d040a98-9fa3-4bf2-ba19-2dbd804740f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a95d34dd28804fc3b45b9488a19047b4, group_id=26009220b25145a1bf4b0da2b68e5b51, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:53.043272 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d040a98-9fa3-4bf2-ba19-2dbd804740f8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:53.062984 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27fcf5-51e6-4312-8e3b-abcfbb039271 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:53.104652 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e297ddc-65bf-4869-a722-580148ef2ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:53.105711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e297ddc-65bf-4869-a722-580148ef2ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:53.106222 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e297ddc-65bf-4869-a722-580148ef2ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:53.106724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e297ddc-65bf-4869-a722-580148ef2ff3 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:53.108167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e297ddc-65bf-4869-a722-580148ef2ff3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:53.108674 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e297ddc-65bf-4869-a722-580148ef2ff3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:53.110372 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e297ddc-65bf-4869-a722-580148ef2ff3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:53.119971 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 205/418] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:53.126884 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e297ddc-65bf-4869-a722-580148ef2ff3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:53.138296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 214/419] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Sep 14 11:45:52 2022] PUT /identity/v3/projects/a95d34dd28804fc3b45b9488a19047b4/groups/26009220b25145a1bf4b0da2b68e5b51/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 11:45:53.249975 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e297ddc-65bf-4869-a722-580148ef2ff3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:53.284165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:53.290821 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8f088-e56e-485d-92e4-1608789b470a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:53.291791 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8f088-e56e-485d-92e4-1608789b470a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:53.292299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8f088-e56e-485d-92e4-1608789b470a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:53.292798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8f088-e56e-485d-92e4-1608789b470a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:53.380017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8f088-e56e-485d-92e4-1608789b470a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:53.381489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8f088-e56e-485d-92e4-1608789b470a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:53.445431 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd80902-0f4e-45de-9cdd-59a0182a827b None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:53.446798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd80902-0f4e-45de-9cdd-59a0182a827b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:53.447978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd80902-0f4e-45de-9cdd-59a0182a827b None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:53.455707 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 215/420] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:53.462900 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b2431-8a66-452c-a7a5-b46740fb5c1a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:53.463515 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b2431-8a66-452c-a7a5-b46740fb5c1a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:53.463994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b2431-8a66-452c-a7a5-b46740fb5c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:53.506981 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 206/421] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:53.518951 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8f088-e56e-485d-92e4-1608789b470a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:53.538961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b64b2431-8a66-452c-a7a5-b46740fb5c1a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:53.540185 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b64b2431-8a66-452c-a7a5-b46740fb5c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:53.602433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:53.699501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 216/422] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:53.723626 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8f088-e56e-485d-92e4-1608789b470a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:53.776424 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce9d72-ba1f-4406-8a50-c727e2e8127e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:53.777080 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce9d72-ba1f-4406-8a50-c727e2e8127e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:53.777368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce9d72-ba1f-4406-8a50-c727e2e8127e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:53.777651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce9d72-ba1f-4406-8a50-c727e2e8127e None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:53.778835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce9d72-ba1f-4406-8a50-c727e2e8127e None admin] RBAC: Authorizing `identity:get_project(project_id=19b75bd757df4c8cac09dd67b1f0ca8d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:53.780979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce9d72-ba1f-4406-8a50-c727e2e8127e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:53.782956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 207/423] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:53 2022] GET /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:53.790966 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b2431-8a66-452c-a7a5-b46740fb5c1a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:53.956004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b2431-8a66-452c-a7a5-b46740fb5c1a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:53.957534 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:53.997445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330b83f-8c7b-4091-9967-1c019a26fe86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:53.998627 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330b83f-8c7b-4091-9967-1c019a26fe86 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:53.998947 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330b83f-8c7b-4091-9967-1c019a26fe86 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:53.999224 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330b83f-8c7b-4091-9967-1c019a26fe86 None admin] PATH_INFO: `/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:54.000340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7330b83f-8c7b-4091-9967-1c019a26fe86 None admin] RBAC: Authorizing `identity:get_group(group_id=bec1848965a34e0bb1bdb91aed87f8f5)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:54.001775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7330b83f-8c7b-4091-9967-1c019a26fe86 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:54.003465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 217/424] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 14 11:45:53 2022] GET /identity/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:54.009413 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce9d72-ba1f-4406-8a50-c727e2e8127e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:54.165283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:54.170531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce9d72-ba1f-4406-8a50-c727e2e8127e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:54.177848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330b83f-8c7b-4091-9967-1c019a26fe86 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:54.214464 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec049e46-de47-4df7-8958-d1c34fae14ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:54.215001 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec049e46-de47-4df7-8958-d1c34fae14ee None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:54.215280 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec049e46-de47-4df7-8958-d1c34fae14ee None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:54.218081 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec049e46-de47-4df7-8958-d1c34fae14ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:54.218816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec049e46-de47-4df7-8958-d1c34fae14ee None admin] RBAC: Adding query filter params (group.id=bec1848965a34e0bb1bdb91aed87f8f5, role.id=d96ee0e293244ae6a04e8ce652aea68e, scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:54.219128 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec049e46-de47-4df7-8958-d1c34fae14ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:54.220548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec049e46-de47-4df7-8958-d1c34fae14ee None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:54.235902 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 208/425] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 14 11:45:54 2022] GET /identity/v3/role_assignments?group.id=bec1848965a34e0bb1bdb91aed87f8f5&scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d&role.id=d96ee0e293244ae6a04e8ce652aea68e => generated 611 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:54.289186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd03bcd-f62c-4a00-9f84-ba7db4061305 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:54.289508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd03bcd-f62c-4a00-9f84-ba7db4061305 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:54.289811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd03bcd-f62c-4a00-9f84-ba7db4061305 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:54.291240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 209/426] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:54.297151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d67e51-e645-44d8-ab0f-9232244ffa1a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:54.297745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d67e51-e645-44d8-ab0f-9232244ffa1a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:54.298265 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d67e51-e645-44d8-ab0f-9232244ffa1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:54.320742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330b83f-8c7b-4091-9967-1c019a26fe86 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:54.347207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-37d67e51-e645-44d8-ab0f-9232244ffa1a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:54.348865 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-37d67e51-e645-44d8-ab0f-9232244ffa1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:54.363001 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f1c35-10f8-47e2-af8f-e442259bd53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:54.363664 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f1c35-10f8-47e2-af8f-e442259bd53d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:54.363946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f1c35-10f8-47e2-af8f-e442259bd53d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:54.364222 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f1c35-10f8-47e2-af8f-e442259bd53d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:54.376806 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f1c35-10f8-47e2-af8f-e442259bd53d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:54.382155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-789f1c35-10f8-47e2-af8f-e442259bd53d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:54.382155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:54.382155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:54.382155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:54.382155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:54.382155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:54.382155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:54.382155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:54.382155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:54.382155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:54.382155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:54.382155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:54.382155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:54.382155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:54.382155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:54.382155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:54.383087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:54.383087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:54.383087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:54.383087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:54.383087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:54.383087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:54.383087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:54.383087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:54.383087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:54.383087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:54.383087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:54.383087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:54.383087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:54.383087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:54.383087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:54.383087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:54.383886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:54.383886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:54.383886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:54.383886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:54.383886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:54.383886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:54.383886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:54.383886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:54.383886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:54.383886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:54.383886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:54.383886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:54.383886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 218/427] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:45:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:45:54.389452 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f1c35-10f8-47e2-af8f-e442259bd53d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:54.430251 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:54.498545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 210/428] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:54.508840 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f1c35-10f8-47e2-af8f-e442259bd53d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:54.544237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85166896-2358-495b-a25e-35b631b1c975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:54.545351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85166896-2358-495b-a25e-35b631b1c975 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:54.545632 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85166896-2358-495b-a25e-35b631b1c975 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:54.545929 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85166896-2358-495b-a25e-35b631b1c975 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:54.546592 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85166896-2358-495b-a25e-35b631b1c975 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:54.546899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85166896-2358-495b-a25e-35b631b1c975 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:54.548266 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85166896-2358-495b-a25e-35b631b1c975 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:54.561471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 219/429] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:45:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:54.563598 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d67e51-e645-44d8-ab0f-9232244ffa1a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:54.657214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:54.666216 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d67e51-e645-44d8-ab0f-9232244ffa1a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:54.701454 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b79c67-5809-4820-b373-e0dce220cb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:54.702115 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b79c67-5809-4820-b373-e0dce220cb7f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:54.702395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b79c67-5809-4820-b373-e0dce220cb7f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:54.702667 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b79c67-5809-4820-b373-e0dce220cb7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:54.774173 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b79c67-5809-4820-b373-e0dce220cb7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:54.775567 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b79c67-5809-4820-b373-e0dce220cb7f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:54.809315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85166896-2358-495b-a25e-35b631b1c975 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:54.883504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:54.897943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 211/430] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:54.901050 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b79c67-5809-4820-b373-e0dce220cb7f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:54.939689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:54.949837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85166896-2358-495b-a25e-35b631b1c975 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:55.005341 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a578953-3282-4607-81d2-724e00cd0b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:55.006026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a578953-3282-4607-81d2-724e00cd0b09 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:55.006311 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a578953-3282-4607-81d2-724e00cd0b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:55.006587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a578953-3282-4607-81d2-724e00cd0b09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:55.023518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a578953-3282-4607-81d2-724e00cd0b09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:55.031765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-4a578953-3282-4607-81d2-724e00cd0b09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:55.031765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:55.031765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:55.031765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:55.031765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:55.031765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:55.031765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:55.031765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:55.031765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:55.031765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:55.031765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:55.031765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:55.031765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:55.031765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:55.031765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:55.031765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:55.032639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:55.032639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:55.032639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:55.032639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:55.032639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:55.032639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:55.032639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:55.032639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:55.032639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:55.032639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:55.032639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:55.032639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:55.032639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:55.032639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:55.032639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:55.032639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:55.033468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:55.033468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:55.033468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:55.033468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:55.033468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:55.033468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:55.033468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:55.033468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:55.033468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:55.033468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:55.033468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 11:45:55.033468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:55.036185 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 220/431] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:55.043516 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a578953-3282-4607-81d2-724e00cd0b09 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:55.046759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b79c67-5809-4820-b373-e0dce220cb7f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:55.100362 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876a7ab-ebc8-4d1c-974d-e75357f1d5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:55.101129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876a7ab-ebc8-4d1c-974d-e75357f1d5a5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:55.101414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876a7ab-ebc8-4d1c-974d-e75357f1d5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:55.101691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876a7ab-ebc8-4d1c-974d-e75357f1d5a5 None admin] PATH_INFO: `/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:55.102899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876a7ab-ebc8-4d1c-974d-e75357f1d5a5 None admin] RBAC: Authorizing `identity:get_project(project_id=19b75bd757df4c8cac09dd67b1f0ca8d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:55.104457 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876a7ab-ebc8-4d1c-974d-e75357f1d5a5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:55.106537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 212/432] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:54 2022] GET /identity/v3/projects/19b75bd757df4c8cac09dd67b1f0ca8d => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:55.112267 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876a7ab-ebc8-4d1c-974d-e75357f1d5a5 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:55.172056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a578953-3282-4607-81d2-724e00cd0b09 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:55.186923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:55.211206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0808e-6c3c-463d-9774-c1b87acfbc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:55.211878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0808e-6c3c-463d-9774-c1b87acfbc16 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:55.212106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0808e-6c3c-463d-9774-c1b87acfbc16 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:55.212388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0808e-6c3c-463d-9774-c1b87acfbc16 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:55.213137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0808e-6c3c-463d-9774-c1b87acfbc16 None admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:55.213451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0808e-6c3c-463d-9774-c1b87acfbc16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:55.214910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0808e-6c3c-463d-9774-c1b87acfbc16 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:55.226563 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 221/433] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:45:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:55.232958 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0808e-6c3c-463d-9774-c1b87acfbc16 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:55.234775 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876a7ab-ebc8-4d1c-974d-e75357f1d5a5 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:55.254255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:55.277692 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49c9fa-1a87-4e7e-b20a-0aea383f381e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:55.278649 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49c9fa-1a87-4e7e-b20a-0aea383f381e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:55.279255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49c9fa-1a87-4e7e-b20a-0aea383f381e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:55.279743 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49c9fa-1a87-4e7e-b20a-0aea383f381e None admin] PATH_INFO: `/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:55.281199 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49c9fa-1a87-4e7e-b20a-0aea383f381e None admin] RBAC: Authorizing `identity:get_group(group_id=bec1848965a34e0bb1bdb91aed87f8f5)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:55.283109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49c9fa-1a87-4e7e-b20a-0aea383f381e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:55.287630 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 213/434] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 14 11:45:55 2022] GET /identity/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:55.292777 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49c9fa-1a87-4e7e-b20a-0aea383f381e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:55.365404 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0808e-6c3c-463d-9774-c1b87acfbc16 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:55.410629 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f9aa6-a5fc-4f69-83d7-a90aa86a7d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:55.411404 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f9aa6-a5fc-4f69-83d7-a90aa86a7d72 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:55.411634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f9aa6-a5fc-4f69-83d7-a90aa86a7d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:55.411915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f9aa6-a5fc-4f69-83d7-a90aa86a7d72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:55.415568 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49c9fa-1a87-4e7e-b20a-0aea383f381e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:55.444801 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:55.453338 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c03a09-5d69-4c44-884a-05cb21aa3c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:55.456490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c03a09-5d69-4c44-884a-05cb21aa3c58 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:55.456490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c03a09-5d69-4c44-884a-05cb21aa3c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:55.456490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c03a09-5d69-4c44-884a-05cb21aa3c58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:55.456490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c03a09-5d69-4c44-884a-05cb21aa3c58 None admin] RBAC: Adding query filter params (group.id=bec1848965a34e0bb1bdb91aed87f8f5, role.id=1ab8c505dc344769abc347f66e529527, scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:55.456490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c03a09-5d69-4c44-884a-05cb21aa3c58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:55.457064 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c03a09-5d69-4c44-884a-05cb21aa3c58 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:55.464794 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 214/435] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 14 11:45:55 2022] GET /identity/v3/role_assignments?group.id=bec1848965a34e0bb1bdb91aed87f8f5&scope.project.id=19b75bd757df4c8cac09dd67b1f0ca8d&role.id=1ab8c505dc344769abc347f66e529527 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:55.466246 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce10ed6-5fb5-4e15-a79e-ab6d5cc72ee9 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:55.467440 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce10ed6-5fb5-4e15-a79e-ab6d5cc72ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:55.468051 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce10ed6-5fb5-4e15-a79e-ab6d5cc72ee9 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:55.469695 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 215/436] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:55.475459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e783f12-144b-4b86-a3c8-e0e777812e3b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:55.475744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e783f12-144b-4b86-a3c8-e0e777812e3b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:55.476021 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e783f12-144b-4b86-a3c8-e0e777812e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:55.496337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1e783f12-144b-4b86-a3c8-e0e777812e3b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:55.497634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1e783f12-144b-4b86-a3c8-e0e777812e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:55.498520 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f9aa6-a5fc-4f69-83d7-a90aa86a7d72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:55.500310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f9aa6-a5fc-4f69-83d7-a90aa86a7d72 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:55.518263 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:55.584657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 216/437] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:45:55.629619 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 222/438] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:45:55.639514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e783f12-144b-4b86-a3c8-e0e777812e3b None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:55.740325 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd8286-71b8-4eb2-bdd3-1bed329b92b1 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:55.740642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd8286-71b8-4eb2-bdd3-1bed329b92b1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:55.740896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd8286-71b8-4eb2-bdd3-1bed329b92b1 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:55.742392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 223/439] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:45:55.747482 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59fafc-1d97-4e3c-ae28-497486ddeeb9 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:55.747796 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59fafc-1d97-4e3c-ae28-497486ddeeb9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:55.748084 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59fafc-1d97-4e3c-ae28-497486ddeeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:55.752665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e783f12-144b-4b86-a3c8-e0e777812e3b None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:55.772439 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ee59fafc-1d97-4e3c-ae28-497486ddeeb9 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:55.773945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ee59fafc-1d97-4e3c-ae28-497486ddeeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:55.786332 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:55.801771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8bfce-2aeb-4efd-91ba-3015fb4d5379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:55.802408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8bfce-2aeb-4efd-91ba-3015fb4d5379 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:55.802696 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8bfce-2aeb-4efd-91ba-3015fb4d5379 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:55.802976 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8bfce-2aeb-4efd-91ba-3015fb4d5379 None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:55.804897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8bfce-2aeb-4efd-91ba-3015fb4d5379 None admin] RBAC: Authorizing `identity:get_project(project_id=2164ed8287f243bc9a4a5347d4dc0b1d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:55.806758 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8bfce-2aeb-4efd-91ba-3015fb4d5379 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:55.808861 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 217/440] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:55 2022] GET /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:55.814984 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8bfce-2aeb-4efd-91ba-3015fb4d5379 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:55.864251 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 224/441] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:45:55.887841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59fafc-1d97-4e3c-ae28-497486ddeeb9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:55.936012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8bfce-2aeb-4efd-91ba-3015fb4d5379 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:55.944904 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:55.977554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24d2ce-7a3f-42d0-a486-396109cf7413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:55.978378 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24d2ce-7a3f-42d0-a486-396109cf7413 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:55.978687 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24d2ce-7a3f-42d0-a486-396109cf7413 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:55.979039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24d2ce-7a3f-42d0-a486-396109cf7413 None admin] PATH_INFO: `/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:55.980381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24d2ce-7a3f-42d0-a486-396109cf7413 None admin] RBAC: Authorizing `identity:get_group(group_id=bec1848965a34e0bb1bdb91aed87f8f5)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:55.981951 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24d2ce-7a3f-42d0-a486-396109cf7413 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:55.983926 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 218/442] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 14 11:45:55 2022] GET /identity/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:55.990950 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24d2ce-7a3f-42d0-a486-396109cf7413 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:56.012460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59fafc-1d97-4e3c-ae28-497486ddeeb9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:56.047330 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3ed80-3114-4af3-be34-97131ea37600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:56.047965 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3ed80-3114-4af3-be34-97131ea37600 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:56.048251 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3ed80-3114-4af3-be34-97131ea37600 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:56.048530 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3ed80-3114-4af3-be34-97131ea37600 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:56.055642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:56.059380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3ed80-3114-4af3-be34-97131ea37600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:56.064242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-07f3ed80-3114-4af3-be34-97131ea37600 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:56.064242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:56.064242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:56.064242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:56.064242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:56.064242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:56.064242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:56.064242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:56.064242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:56.064242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:56.064242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:56.064242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:56.064242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:56.064242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:56.064242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:56.064242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:56.066901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:56.066901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:56.066901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:56.066901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:56.066901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:56.066901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:56.066901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:56.066901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:56.066901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:56.066901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:56.066901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:56.066901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:56.066901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:56.066901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:56.066901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:56.066901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:56.067718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:56.067718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:56.067718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:56.067718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:56.067718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:56.067718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:56.067718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:56.067718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:56.067718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:56.067718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:56.067718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:45:56.067718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:45:56.067718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 225/443] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:45:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:45:56.071102 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3ed80-3114-4af3-be34-97131ea37600 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:56.104768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24d2ce-7a3f-42d0-a486-396109cf7413 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:56.139644 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffc820-0be3-42b8-894d-9de2bc239694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:56.140170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffc820-0be3-42b8-894d-9de2bc239694 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:56.140450 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffc820-0be3-42b8-894d-9de2bc239694 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:56.140731 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffc820-0be3-42b8-894d-9de2bc239694 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:56.141448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffc820-0be3-42b8-894d-9de2bc239694 None admin] RBAC: Adding query filter params (group.id=bec1848965a34e0bb1bdb91aed87f8f5, role.id=d96ee0e293244ae6a04e8ce652aea68e, scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:56.141756 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffc820-0be3-42b8-894d-9de2bc239694 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:56.143195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffc820-0be3-42b8-894d-9de2bc239694 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:56.148791 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 219/444] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 14 11:45:55 2022] GET /identity/v3/role_assignments?group.id=bec1848965a34e0bb1bdb91aed87f8f5&scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d&role.id=d96ee0e293244ae6a04e8ce652aea68e => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:56.149549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffc820-0be3-42b8-894d-9de2bc239694 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:56.202350 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3ed80-3114-4af3-be34-97131ea37600 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:56.204922 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:56.240682 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565db46f-fbb1-4822-ad59-839593f584cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:56.242057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565db46f-fbb1-4822-ad59-839593f584cf None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:56.242343 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565db46f-fbb1-4822-ad59-839593f584cf None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:56.242617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565db46f-fbb1-4822-ad59-839593f584cf None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:56.243302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565db46f-fbb1-4822-ad59-839593f584cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:56.243671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565db46f-fbb1-4822-ad59-839593f584cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:56.245080 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565db46f-fbb1-4822-ad59-839593f584cf None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:56.254644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 226/445] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:45:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:56.260511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565db46f-fbb1-4822-ad59-839593f584cf None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:56.272005 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffc820-0be3-42b8-894d-9de2bc239694 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:56.301884 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:56.314834 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0960336-db41-4b73-90b8-9b827135931f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:56.315481 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0960336-db41-4b73-90b8-9b827135931f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:56.315759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0960336-db41-4b73-90b8-9b827135931f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:56.316102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0960336-db41-4b73-90b8-9b827135931f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:56.329236 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0960336-db41-4b73-90b8-9b827135931f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:45:56.335017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-f0960336-db41-4b73-90b8-9b827135931f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:45:56.335017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:45:56.335017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:45:56.335017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:45:56.335017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:45:56.335017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:45:56.335017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:45:56.335017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:45:56.335017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:45:56.335017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:45:56.335017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:45:56.335017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:45:56.335017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:45:56.335017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:45:56.335017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:45:56.335017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:45:56.335937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:45:56.335937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:45:56.335937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:45:56.335937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:45:56.335937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:45:56.335937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:45:56.335937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:45:56.335937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:45:56.335937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:45:56.335937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:45:56.335937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:45:56.335937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:45:56.335937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:45:56.335937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:45:56.335937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:45:56.335937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:45:56.336747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:45:56.336747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:45:56.336747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:45:56.336747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:45:56.336747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:45:56.336747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:45:56.336747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:45:56.336747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:45:56.336747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:45:56.336747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:45:56.336747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:45:56.336747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:45:56.341647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 220/446] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:45:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:45:56.342872 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0960336-db41-4b73-90b8-9b827135931f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:56.378522 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565db46f-fbb1-4822-ad59-839593f584cf None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:56.418846 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6c64a-5c43-4dd9-89b4-c17c0ca8f2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:56.420175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6c64a-5c43-4dd9-89b4-c17c0ca8f2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:56.420460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6c64a-5c43-4dd9-89b4-c17c0ca8f2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:56.420741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6c64a-5c43-4dd9-89b4-c17c0ca8f2f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:56.457828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:56.469194 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0960336-db41-4b73-90b8-9b827135931f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:56.499930 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6c64a-5c43-4dd9-89b4-c17c0ca8f2f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:56.501405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6c64a-5c43-4dd9-89b4-c17c0ca8f2f6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:56.508488 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d13dfb-1727-453d-9524-d021f7efe502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:56.509106 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d13dfb-1727-453d-9524-d021f7efe502 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:56.509390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d13dfb-1727-453d-9524-d021f7efe502 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:56.509668 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d13dfb-1727-453d-9524-d021f7efe502 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:56.510377 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d13dfb-1727-453d-9524-d021f7efe502 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:56.510687 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d13dfb-1727-453d-9524-d021f7efe502 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:56.512054 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d13dfb-1727-453d-9524-d021f7efe502 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:56.521829 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 221/447] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:45:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:56.527537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d13dfb-1727-453d-9524-d021f7efe502 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:56.576025 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:56.610497 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 227/448] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 11:45:56.616096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6c64a-5c43-4dd9-89b4-c17c0ca8f2f6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:56.631759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d13dfb-1727-453d-9524-d021f7efe502 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:56.664526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa221cf0-dd3a-4611-8a38-7f324688824a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:56.665129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa221cf0-dd3a-4611-8a38-7f324688824a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:56.665414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa221cf0-dd3a-4611-8a38-7f324688824a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:56.665692 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa221cf0-dd3a-4611-8a38-7f324688824a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:56.678915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:56.711629 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6c64a-5c43-4dd9-89b4-c17c0ca8f2f6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:56.734219 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa221cf0-dd3a-4611-8a38-7f324688824a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:56.735677 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa221cf0-dd3a-4611-8a38-7f324688824a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:56.744433 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddc1e0-ceac-4540-8489-7b85550bb34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:56.745048 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddc1e0-ceac-4540-8489-7b85550bb34b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:56.745325 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddc1e0-ceac-4540-8489-7b85550bb34b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:56.745598 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddc1e0-ceac-4540-8489-7b85550bb34b None admin] PATH_INFO: `/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:56.746739 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaddc1e0-ceac-4540-8489-7b85550bb34b None admin] RBAC: Authorizing `identity:get_project(project_id=2164ed8287f243bc9a4a5347d4dc0b1d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:56.748722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaddc1e0-ceac-4540-8489-7b85550bb34b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:56.750358 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 228/449] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:56 2022] GET /identity/v3/projects/2164ed8287f243bc9a4a5347d4dc0b1d => generated 302 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:56.755932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddc1e0-ceac-4540-8489-7b85550bb34b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:56.780072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:56.835621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 222/450] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:45:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 11:45:56.841125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa221cf0-dd3a-4611-8a38-7f324688824a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:56.855498 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddc1e0-ceac-4540-8489-7b85550bb34b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:56.889420 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c8891-8df0-438d-beed-8ee095b8ff60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:56.890067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c8891-8df0-438d-beed-8ee095b8ff60 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:56.890346 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c8891-8df0-438d-beed-8ee095b8ff60 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:56.890625 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c8891-8df0-438d-beed-8ee095b8ff60 None admin] PATH_INFO: `/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:56.891725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c8891-8df0-438d-beed-8ee095b8ff60 None admin] RBAC: Authorizing `identity:get_group(group_id=bec1848965a34e0bb1bdb91aed87f8f5)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:56.893151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c8891-8df0-438d-beed-8ee095b8ff60 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:56.895025 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 229/451] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 14 11:45:56 2022] GET /identity/v3/groups/bec1848965a34e0bb1bdb91aed87f8f5 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:56.900825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c8891-8df0-438d-beed-8ee095b8ff60 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:56.903399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:56.937746 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa221cf0-dd3a-4611-8a38-7f324688824a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:56.972939 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ba457-9907-475e-92a2-b93a18cca8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:56.973578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ba457-9907-475e-92a2-b93a18cca8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:56.973885 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ba457-9907-475e-92a2-b93a18cca8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:56.974168 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ba457-9907-475e-92a2-b93a18cca8d1 None admin] PATH_INFO: `/v3/projects/a95d34dd28804fc3b45b9488a19047b4` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:56.975706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ba457-9907-475e-92a2-b93a18cca8d1 None admin] RBAC: Authorizing `identity:get_project(project_id=a95d34dd28804fc3b45b9488a19047b4)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:56.977505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ba457-9907-475e-92a2-b93a18cca8d1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:56.979551 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 223/452] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:45:56 2022] GET /identity/v3/projects/a95d34dd28804fc3b45b9488a19047b4 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:56.985503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ba457-9907-475e-92a2-b93a18cca8d1 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:56.999509 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:57.000194 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c8891-8df0-438d-beed-8ee095b8ff60 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:57.033750 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfac181-98dd-4851-9923-685a0507cbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:57.034379 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfac181-98dd-4851-9923-685a0507cbfa None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:57.034665 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfac181-98dd-4851-9923-685a0507cbfa None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:57.034941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfac181-98dd-4851-9923-685a0507cbfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:57.035658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfac181-98dd-4851-9923-685a0507cbfa None admin] RBAC: Adding query filter params (group.id=bec1848965a34e0bb1bdb91aed87f8f5, role.id=1ab8c505dc344769abc347f66e529527, scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:57.035967 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfac181-98dd-4851-9923-685a0507cbfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:57.037418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfac181-98dd-4851-9923-685a0507cbfa None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:57.043038 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 230/453] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 14 11:45:56 2022] GET /identity/v3/role_assignments?group.id=bec1848965a34e0bb1bdb91aed87f8f5&scope.project.id=2164ed8287f243bc9a4a5347d4dc0b1d&role.id=1ab8c505dc344769abc347f66e529527 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:57.085678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ba457-9907-475e-92a2-b93a18cca8d1 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:57.122396 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2567f9-5912-4f8a-9d84-364a80a3ba86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:57.123084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2567f9-5912-4f8a-9d84-364a80a3ba86 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:57.123370 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2567f9-5912-4f8a-9d84-364a80a3ba86 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:57.123651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2567f9-5912-4f8a-9d84-364a80a3ba86 None admin] PATH_INFO: `/v3/groups/26009220b25145a1bf4b0da2b68e5b51` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:57.124821 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2567f9-5912-4f8a-9d84-364a80a3ba86 None admin] RBAC: Authorizing `identity:get_group(group_id=26009220b25145a1bf4b0da2b68e5b51)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:57.126362 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2567f9-5912-4f8a-9d84-364a80a3ba86 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:57.128013 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 224/454] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 14 11:45:56 2022] GET /identity/v3/groups/26009220b25145a1bf4b0da2b68e5b51 => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:45:57.134111 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfac181-98dd-4851-9923-685a0507cbfa None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:45:57.202649 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:45:57.238591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfac181-98dd-4851-9923-685a0507cbfa None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:45:57.272483 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dda96e-bc06-4503-9493-efd9314ec09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:45:57.273891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dda96e-bc06-4503-9493-efd9314ec09f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:57.274182 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dda96e-bc06-4503-9493-efd9314ec09f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:57.274466 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dda96e-bc06-4503-9493-efd9314ec09f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:57.275246 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dda96e-bc06-4503-9493-efd9314ec09f None admin] RBAC: Adding query filter params (group.id=26009220b25145a1bf4b0da2b68e5b51, role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.project.id=a95d34dd28804fc3b45b9488a19047b4) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:45:57.275558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dda96e-bc06-4503-9493-efd9314ec09f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:45:57.277018 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dda96e-bc06-4503-9493-efd9314ec09f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:45:57.283948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 231/455] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 14 11:45:57 2022] GET /identity/v3/role_assignments?group.id=26009220b25145a1bf4b0da2b68e5b51&scope.project.id=a95d34dd28804fc3b45b9488a19047b4&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:45:59.786786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b8aac-b4b5-4a03-85bb-bdeae3c34f3f None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:59.787337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b8aac-b4b5-4a03-85bb-bdeae3c34f3f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:59.787337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b8aac-b4b5-4a03-85bb-bdeae3c34f3f None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:59.788901 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 225/456] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:59.794341 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13647e7-9dd7-4d64-b6e7-b84eb4dc4d39 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:59.794767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13647e7-9dd7-4d64-b6e7-b84eb4dc4d39 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:59.794948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13647e7-9dd7-4d64-b6e7-b84eb4dc4d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:59.814612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e13647e7-9dd7-4d64-b6e7-b84eb4dc4d39 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:59.815860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e13647e7-9dd7-4d64-b6e7-b84eb4dc4d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:45:59.917499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 232/457] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:45:59.939573 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bab906-37f6-4c3d-8235-0ee249f7add8 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:59.939851 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bab906-37f6-4c3d-8235-0ee249f7add8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:59.940135 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bab906-37f6-4c3d-8235-0ee249f7add8 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:59.941471 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 226/458] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:45:59.946865 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff64c2d-810c-4350-b729-d2d6b129f7d6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:45:59.947154 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff64c2d-810c-4350-b729-d2d6b129f7d6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:45:59.947439 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff64c2d-810c-4350-b729-d2d6b129f7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:45:59.971283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-cff64c2d-810c-4350-b729-d2d6b129f7d6 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:45:59.972546 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cff64c2d-810c-4350-b729-d2d6b129f7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:00.052203 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:00.070904 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 233/459] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:00.212464 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833ea6f-665e-4fb5-95a5-7e1826701f56 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:00.212817 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833ea6f-665e-4fb5-95a5-7e1826701f56 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:00.213010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833ea6f-665e-4fb5-95a5-7e1826701f56 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:00.214624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 227/460] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:00.220584 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833ea6f-665e-4fb5-95a5-7e1826701f56 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:00.222263 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b902d7-db92-4bf6-a980-c0d5834069e6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:00.222552 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b902d7-db92-4bf6-a980-c0d5834069e6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:00.222832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b902d7-db92-4bf6-a980-c0d5834069e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:00.255363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d6b902d7-db92-4bf6-a980-c0d5834069e6 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:00.258740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d6b902d7-db92-4bf6-a980-c0d5834069e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:00.359262 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:00.384869 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 234/461] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:00.391672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833ea6f-665e-4fb5-95a5-7e1826701f56 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:00.427826 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a245c-e72d-4e18-9f90-2de7724ded8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:00.428472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a245c-e72d-4e18-9f90-2de7724ded8f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:00.428764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a245c-e72d-4e18-9f90-2de7724ded8f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:00.429049 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a245c-e72d-4e18-9f90-2de7724ded8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:00.440252 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a245c-e72d-4e18-9f90-2de7724ded8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:00.441738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a245c-e72d-4e18-9f90-2de7724ded8f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:00.457371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-2a0a245c-e72d-4e18-9f90-2de7724ded8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:00.457371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:00.457371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:00.457371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:00.457371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:00.457371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:00.457371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:00.457371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:00.457371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:00.457371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:00.457371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:00.457371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:00.457371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:00.457371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:00.457371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:00.457371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:00.458354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:00.458354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:00.458354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:00.458354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:00.458354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:00.458354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:00.458354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:00.458354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:00.458354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:00.458354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:00.458354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:00.458354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:00.458354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:00.458354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:00.458354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:00.458354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:00.458354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:00.459594 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:00.459594 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:00.459594 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:00.459594 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 228/462] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:00.464845 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b902d7-db92-4bf6-a980-c0d5834069e6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:00.475215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a245c-e72d-4e18-9f90-2de7724ded8f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:00.651239 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a245c-e72d-4e18-9f90-2de7724ded8f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:00.664808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b902d7-db92-4bf6-a980-c0d5834069e6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:00.702161 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:00.705511 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:00.718005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dba174-6185-4cac-b1ec-d24d8f3850b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:00.718720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dba174-6185-4cac-b1ec-d24d8f3850b5 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:00.726452 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dba174-6185-4cac-b1ec-d24d8f3850b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:00.726452 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dba174-6185-4cac-b1ec-d24d8f3850b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:00.726452 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dba174-6185-4cac-b1ec-d24d8f3850b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:00.726672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88268c7c-147f-447c-a9dc-3c592dd84280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:00.726758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dba174-6185-4cac-b1ec-d24d8f3850b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:00.726842 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88268c7c-147f-447c-a9dc-3c592dd84280 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:00.726842 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88268c7c-147f-447c-a9dc-3c592dd84280 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:00.726842 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88268c7c-147f-447c-a9dc-3c592dd84280 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:00.727013 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dba174-6185-4cac-b1ec-d24d8f3850b5 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:00.743371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88268c7c-147f-447c-a9dc-3c592dd84280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:00.744847 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88268c7c-147f-447c-a9dc-3c592dd84280 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:00.746854 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 235/463] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:00.748308 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dba174-6185-4cac-b1ec-d24d8f3850b5 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:00.762441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-88268c7c-147f-447c-a9dc-3c592dd84280 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:00.762441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:00.762441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:00.762441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:00.762441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:00.762441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:00.762441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:00.762441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:00.762441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:00.762441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:00.762441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:00.762441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:00.762441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:00.762441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:00.762441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:00.762441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:00.769132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:00.769132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:00.769132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:00.769132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:00.769132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:00.769132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:00.769132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:00.769132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:00.769132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:00.769132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:00.769132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:00.769132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:00.769132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:00.769132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:00.769132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:00.769132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:00.769132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:00.770343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:00.770343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:00.770343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:00.770343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 229/464] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:00.770343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88268c7c-147f-447c-a9dc-3c592dd84280 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:00.858486 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dba174-6185-4cac-b1ec-d24d8f3850b5 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:00.874856 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88268c7c-147f-447c-a9dc-3c592dd84280 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:00.896597 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a376d-687f-46a2-a7d3-3a7256845f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:00.897513 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a376d-687f-46a2-a7d3-3a7256845f1f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:00.897984 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a376d-687f-46a2-a7d3-3a7256845f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:00.898411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a376d-687f-46a2-a7d3-3a7256845f1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:00.910641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a376d-687f-46a2-a7d3-3a7256845f1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:00.912373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a376d-687f-46a2-a7d3-3a7256845f1f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:00.916845 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f460a2-c69d-4b2d-b22a-ab7b1052b3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:00.917539 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f460a2-c69d-4b2d-b22a-ab7b1052b3ce None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:00.917831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f460a2-c69d-4b2d-b22a-ab7b1052b3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:00.918136 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f460a2-c69d-4b2d-b22a-ab7b1052b3ce None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:00.918808 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f460a2-c69d-4b2d-b22a-ab7b1052b3ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:00.920312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f460a2-c69d-4b2d-b22a-ab7b1052b3ce None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:00.930366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-520a376d-687f-46a2-a7d3-3a7256845f1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:00.930366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:00.930366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:00.930366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:00.930366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:00.930366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:00.930366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:00.930366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:00.930366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:00.930366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:00.930366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:00.930366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:00.930366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:00.930366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:00.930366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:00.930366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:00.931447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:00.931447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:00.931447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:00.931447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:00.931447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:00.931447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:00.931447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:00.931447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:00.931447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:00.931447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:00.931447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:00.931447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:00.931447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:00.931447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:00.931447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:00.931447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:00.931447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:00.932360 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:00.932360 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:00.932360 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:00.933751 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 236/465] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:00.934694 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a376d-687f-46a2-a7d3-3a7256845f1f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:00.973311 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:01.017662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 230/466] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 14 11:46:00 2022] POST /identity/v3/users => generated 246 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:46:01.018907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f460a2-c69d-4b2d-b22a-ab7b1052b3ce None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:01.048494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a376d-687f-46a2-a7d3-3a7256845f1f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:01.081193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:01.089352 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd4ba8-49da-4d62-819f-1f75ce0eb5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:01.090491 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd4ba8-49da-4d62-819f-1f75ce0eb5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:01.090974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd4ba8-49da-4d62-819f-1f75ce0eb5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:01.094729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd4ba8-49da-4d62-819f-1f75ce0eb5d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:01.094729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd4ba8-49da-4d62-819f-1f75ce0eb5d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:01.094729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd4ba8-49da-4d62-819f-1f75ce0eb5d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:01.094729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd4ba8-49da-4d62-819f-1f75ce0eb5d8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:01.108091 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 237/467] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:01.109467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86c5ae-0ae0-4792-adcc-bb5972a37f41 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:01.109759 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86c5ae-0ae0-4792-adcc-bb5972a37f41 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:01.110078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86c5ae-0ae0-4792-adcc-bb5972a37f41 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:01.115561 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 238/468] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:01.115561 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86c5ae-0ae0-4792-adcc-bb5972a37f41 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:01.162560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f460a2-c69d-4b2d-b22a-ab7b1052b3ce None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:01.211957 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191c0c5-6827-4451-8be7-6a1bf21acec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:01.212529 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191c0c5-6827-4451-8be7-6a1bf21acec5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:01.212815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191c0c5-6827-4451-8be7-6a1bf21acec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:01.213090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191c0c5-6827-4451-8be7-6a1bf21acec5 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:01.213893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b191c0c5-6827-4451-8be7-6a1bf21acec5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:01.214408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b191c0c5-6827-4451-8be7-6a1bf21acec5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:01.215716 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b191c0c5-6827-4451-8be7-6a1bf21acec5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:01.225407 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:01.234790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 231/469] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:01.236508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0455b97-9e21-4311-955d-ec316dcdd24f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:01.243147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0455b97-9e21-4311-955d-ec316dcdd24f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:01.243147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0455b97-9e21-4311-955d-ec316dcdd24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:01.243352 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86c5ae-0ae0-4792-adcc-bb5972a37f41 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:01.258933 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e0455b97-9e21-4311-955d-ec316dcdd24f None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:01.260163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e0455b97-9e21-4311-955d-ec316dcdd24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:01.300835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbde379-b4bb-45fc-b50c-d271ab2bab01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:01.301800 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbde379-b4bb-45fc-b50c-d271ab2bab01 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:01.303332 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbde379-b4bb-45fc-b50c-d271ab2bab01 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:01.303816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbde379-b4bb-45fc-b50c-d271ab2bab01 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:01.304986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbde379-b4bb-45fc-b50c-d271ab2bab01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:01.307666 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbde379-b4bb-45fc-b50c-d271ab2bab01 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:01.346996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 232/470] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:01.348137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0455b97-9e21-4311-955d-ec316dcdd24f None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:01.381511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 239/471] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 14 11:46:01 2022] POST /identity/v3/users => generated 248 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 11:46:01.393905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:01.477968 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0455b97-9e21-4311-955d-ec316dcdd24f None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:01.517438 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f6f20-0f50-4a47-b4c2-c80cd227aca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:01.518473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f6f20-0f50-4a47-b4c2-c80cd227aca4 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:01.518938 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f6f20-0f50-4a47-b4c2-c80cd227aca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:01.519389 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f6f20-0f50-4a47-b4c2-c80cd227aca4 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:01.520265 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f6f20-0f50-4a47-b4c2-c80cd227aca4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:01.521917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f6f20-0f50-4a47-b4c2-c80cd227aca4 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:01.586610 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 233/472] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 14 11:46:01 2022] POST /identity/v3/users => generated 248 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:46:01.674154 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e53d6-fca6-4bbb-9bfc-a4baf9eadb2a None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:01.674896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e53d6-fca6-4bbb-9bfc-a4baf9eadb2a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:01.675367 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e53d6-fca6-4bbb-9bfc-a4baf9eadb2a None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:01.678199 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 240/473] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:01.683659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5359e-a286-4260-b6b7-59f2cb110b8e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:01.683955 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5359e-a286-4260-b6b7-59f2cb110b8e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:01.684223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5359e-a286-4260-b6b7-59f2cb110b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:01.708527 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1fb5359e-a286-4260-b6b7-59f2cb110b8e None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:01.709763 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1fb5359e-a286-4260-b6b7-59f2cb110b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:01.791111 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e53d6-fca6-4bbb-9bfc-a4baf9eadb2a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:01.797898 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 234/474] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:46:01.933219 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e53d6-fca6-4bbb-9bfc-a4baf9eadb2a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:01.969192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fbe68-0328-4901-947b-eedaf0955b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:01.970130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fbe68-0328-4901-947b-eedaf0955b1b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:01.970554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fbe68-0328-4901-947b-eedaf0955b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:01.970967 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fbe68-0328-4901-947b-eedaf0955b1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:01.981784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473fbe68-0328-4901-947b-eedaf0955b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:01.984095 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473fbe68-0328-4901-947b-eedaf0955b1b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:01.992520 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:01.999571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-473fbe68-0328-4901-947b-eedaf0955b1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:01.999571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:01.999571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:01.999571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:01.999571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:01.999571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:01.999571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:01.999571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:01.999571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:01.999571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:01.999571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:01.999571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:01.999571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:01.999571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:01.999571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:01.999571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:02.006415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:02.006415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:02.006415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:02.006415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:02.006415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:02.006415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:02.006415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:02.006415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:02.006415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:02.006415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:02.006415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:02.006415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:02.006415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:02.006415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:02.006415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:02.006415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:02.006415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:02.007477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:02.007477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:02.007477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:02.007733 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 241/475] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:02.010966 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5359e-a286-4260-b6b7-59f2cb110b8e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:02.045679 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:02.150017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5359e-a286-4260-b6b7-59f2cb110b8e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:02.194056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fbe68-0328-4901-947b-eedaf0955b1b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:02.197815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa599e6b-7213-4926-a01b-e20ea3682381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:02.197815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa599e6b-7213-4926-a01b-e20ea3682381 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:02.197815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa599e6b-7213-4926-a01b-e20ea3682381 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:02.197815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa599e6b-7213-4926-a01b-e20ea3682381 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:02.198275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa599e6b-7213-4926-a01b-e20ea3682381 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:02.198275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa599e6b-7213-4926-a01b-e20ea3682381 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:02.198528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa599e6b-7213-4926-a01b-e20ea3682381 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:02.221590 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 235/476] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:02.227512 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa599e6b-7213-4926-a01b-e20ea3682381 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:02.342623 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:02.367014 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa599e6b-7213-4926-a01b-e20ea3682381 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:02.395890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fbe68-0328-4901-947b-eedaf0955b1b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:02.405839 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713476c-8609-4011-a6ac-529caaeb6297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:02.406759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713476c-8609-4011-a6ac-529caaeb6297 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:02.407233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713476c-8609-4011-a6ac-529caaeb6297 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:02.407709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713476c-8609-4011-a6ac-529caaeb6297 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:02.408665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713476c-8609-4011-a6ac-529caaeb6297 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:02.410519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713476c-8609-4011-a6ac-529caaeb6297 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:02.445804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e4c99-24b9-43e4-a471-3133b5121572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:02.446435 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e4c99-24b9-43e4-a471-3133b5121572 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:02.446721 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e4c99-24b9-43e4-a471-3133b5121572 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:02.447001 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e4c99-24b9-43e4-a471-3133b5121572 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:02.447748 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e4c99-24b9-43e4-a471-3133b5121572 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:02.449142 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e4c99-24b9-43e4-a471-3133b5121572 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:02.477596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.sql.core [None req-dc1e4c99-24b9-43e4-a471-3133b5121572 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 14 11:46:02.477596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 14 11:46:02.477596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [parameters: {'id': '54718943a31045b1a5e8ec110b28e9f8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 14 11:46:02.477596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81581) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 14 11:46:02.496087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-dc1e4c99-24b9-43e4-a471-3133b5121572 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 14 11:46:02.496087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:02.496087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 11:46:02.496087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 11:46:02.496087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 11:46:02.496087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 11:46:02.496087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 11:46:02.496087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 11:46:02.496087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 11:46:02.496087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application conn.query(q) Sep 14 11:46:02.496087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 11:46:02.496087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 11:46:02.496087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 11:46:02.496087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result.read() Sep 14 11:46:02.496087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 11:46:02.496999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 11:46:02.496999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 11:46:02.496999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 11:46:02.496999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 11:46:02.496999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 11:46:02.496999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 11:46:02.496999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 11:46:02.496999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 14 11:46:02.496999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:02.496999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 14 11:46:02.496999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:02.496999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:02.496999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 14 11:46:02.496999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 14 11:46:02.496999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 14 11:46:02.496999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return ref.to_dict() Sep 14 11:46:02.496999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 11:46:02.496999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application next(self.gen) Sep 14 11:46:02.498003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 14 11:46:02.498003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application yield resource Sep 14 11:46:02.498003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 11:46:02.498003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application next(self.gen) Sep 14 11:46:02.498003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 14 11:46:02.498003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.session.rollback() Sep 14 11:46:02.498003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 14 11:46:02.498003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.force_reraise() Sep 14 11:46:02.498003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 14 11:46:02.498003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise self.value Sep 14 11:46:02.498003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 14 11:46:02.498003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 14 11:46:02.498003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 14 11:46:02.498003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application session.commit() Sep 14 11:46:02.498003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 14 11:46:02.498003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 14 11:46:02.498003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 14 11:46:02.499469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._prepare_impl() Sep 14 11:46:02.499469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 14 11:46:02.499469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.session.flush() Sep 14 11:46:02.499469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 14 11:46:02.499469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._flush(objects) Sep 14 11:46:02.499469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 14 11:46:02.499469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 14 11:46:02.499469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 14 11:46:02.499469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application compat.raise_( Sep 14 11:46:02.499469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 11:46:02.499469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception Sep 14 11:46:02.499469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 14 11:46:02.499469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application flush_context.execute() Sep 14 11:46:02.499469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 14 11:46:02.499469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rec.execute(self) Sep 14 11:46:02.499469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 14 11:46:02.499469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 14 11:46:02.500351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 14 11:46:02.500351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 14 11:46:02.500351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 14 11:46:02.500351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 14 11:46:02.500351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 14 11:46:02.500351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 14 11:46:02.500351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 14 11:46:02.500351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 14 11:46:02.500351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 14 11:46:02.500351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 14 11:46:02.500351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 14 11:46:02.500351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 14 11:46:02.500351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 14 11:46:02.500351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 14 11:46:02.500351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 11:46:02.500351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception Sep 14 11:46:02.501188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 11:46:02.501188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 11:46:02.501188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 11:46:02.501188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 11:46:02.501188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 11:46:02.501188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 11:46:02.501188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 11:46:02.501188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application conn.query(q) Sep 14 11:46:02.501188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 11:46:02.501188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 11:46:02.501188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 11:46:02.501188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result.read() Sep 14 11:46:02.501188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 11:46:02.501188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 11:46:02.501188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 11:46:02.501188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 11:46:02.502004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 11:46:02.502004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 11:46:02.502004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 11:46:02.502004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 11:46:02.502004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 14 11:46:02.502004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 14 11:46:02.502004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [parameters: {'id': '54718943a31045b1a5e8ec110b28e9f8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 14 11:46:02.502004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 14 11:46:02.502004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:02.502004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:02.502004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:02.502004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:02.502004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:02.502004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:02.502004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:02.502004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:02.504613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:02.504613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:02.504613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:02.504613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:02.504613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:02.504613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:02.504613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 14 11:46:02.504613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 14 11:46:02.504613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:02.504613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:02.504613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 14 11:46:02.504613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 14 11:46:02.504613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 14 11:46:02.504613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 14 11:46:02.504613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 14 11:46:02.504613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:02.512398 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 242/477] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Sep 14 11:46:02 2022] POST /identity/v3/roles => generated 144 bytes in 309 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:02.512398 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e4c99-24b9-43e4-a471-3133b5121572 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:02.522033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 236/478] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 14 11:46:02 2022] POST /identity/v3/users => generated 249 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:46:02.642337 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:02.656320 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e4c99-24b9-43e4-a471-3133b5121572 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:02.723357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e573d7-7d7a-4c5b-bcfe-22b7dfcceaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:02.724400 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e573d7-7d7a-4c5b-bcfe-22b7dfcceaa8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:02.724887 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e573d7-7d7a-4c5b-bcfe-22b7dfcceaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:02.725359 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e573d7-7d7a-4c5b-bcfe-22b7dfcceaa8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:02.737402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e573d7-7d7a-4c5b-bcfe-22b7dfcceaa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:02.742911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-93e573d7-7d7a-4c5b-bcfe-22b7dfcceaa8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:46:02.742911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:02.742911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:02.742911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:02.742911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:02.742911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:02.742911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:02.742911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:02.742911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:02.742911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:02.742911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:02.742911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:02.742911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:02.742911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:02.742911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:02.742911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:02.743812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:02.743812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:02.743812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:02.743812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:02.743812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:02.743812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:02.743812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:02.743812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:02.743812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:02.743812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:02.743812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:02.743812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:02.743812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:02.743812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:02.743812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:02.743812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:02.744619 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:02.744619 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:02.744619 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:02.744619 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:02.744619 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:02.744619 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:02.744619 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:02.744619 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:02.744619 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:02.744619 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:02.744619 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 11:46:02.744619 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:02.747616 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 243/479] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:46:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:02.753200 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713476c-8609-4011-a6ac-529caaeb6297 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:02.864320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713476c-8609-4011-a6ac-529caaeb6297 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:02.893824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:02.903555 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb7258-aba4-4ada-8980-8f3ddf86bb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:02.904722 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb7258-aba4-4ada-8980-8f3ddf86bb41 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:02.905009 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb7258-aba4-4ada-8980-8f3ddf86bb41 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:02.910061 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb7258-aba4-4ada-8980-8f3ddf86bb41 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:02.910061 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb7258-aba4-4ada-8980-8f3ddf86bb41 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:02.910061 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb7258-aba4-4ada-8980-8f3ddf86bb41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:02.910061 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb7258-aba4-4ada-8980-8f3ddf86bb41 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:02.918751 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 237/480] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:03.719245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022ebb7-f4c4-43dd-bd4d-6877fbf2eece None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:03.719760 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022ebb7-f4c4-43dd-bd4d-6877fbf2eece None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:03.719835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022ebb7-f4c4-43dd-bd4d-6877fbf2eece None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:03.721401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 244/481] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:03.726759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab3aed-4850-42e4-809e-01f48112329b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:03.727170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab3aed-4850-42e4-809e-01f48112329b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:03.727380 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab3aed-4850-42e4-809e-01f48112329b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:03.746247 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-91ab3aed-4850-42e4-809e-01f48112329b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:03.747436 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-91ab3aed-4850-42e4-809e-01f48112329b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:03.840429 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 238/482] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:04.219084 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce72c2-7e54-4b64-90ec-f3cbda350013 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:04.219826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce72c2-7e54-4b64-90ec-f3cbda350013 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:04.220314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce72c2-7e54-4b64-90ec-f3cbda350013 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:04.222439 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 245/483] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:04.222907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab3aed-4850-42e4-809e-01f48112329b None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:04.229272 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a798ff-12a5-48ac-947d-96c7b02f3127 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:04.229650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a798ff-12a5-48ac-947d-96c7b02f3127 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:04.229834 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a798ff-12a5-48ac-947d-96c7b02f3127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:04.270506 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-55a798ff-12a5-48ac-947d-96c7b02f3127 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:04.272609 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-55a798ff-12a5-48ac-947d-96c7b02f3127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:04.400142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab3aed-4850-42e4-809e-01f48112329b None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:04.410953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 246/484] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:46:04.428358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:04.438598 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7da3b3-4d26-4ae8-ad2d-71ccaee6b2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:04.439236 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7da3b3-4d26-4ae8-ad2d-71ccaee6b2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:04.439520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7da3b3-4d26-4ae8-ad2d-71ccaee6b2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:04.439796 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7da3b3-4d26-4ae8-ad2d-71ccaee6b2b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:04.449983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7da3b3-4d26-4ae8-ad2d-71ccaee6b2b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:04.455642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-6d7da3b3-4d26-4ae8-ad2d-71ccaee6b2b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:04.455642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:04.455642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:04.455642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:04.455642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:04.455642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:04.455642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:04.455642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:04.455642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:04.455642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:04.455642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:04.455642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:04.455642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:04.455642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:04.455642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:04.455642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:04.456538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:04.456538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:04.456538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:04.456538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:04.456538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:04.456538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:04.456538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:04.456538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:04.456538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:04.456538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:04.456538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:04.456538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:04.456538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:04.456538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:04.456538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:04.456538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:04.457767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:04.457767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:04.457767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:04.457767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:04.457767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:04.457767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:04.457767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:04.457767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:04.457767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:04.457767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:04.457767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:04.457767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:04.457767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 239/485] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:04.462771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a798ff-12a5-48ac-947d-96c7b02f3127 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:04.493893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75eba61-411f-4e90-9570-96b20c9dfc44 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:04.494160 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75eba61-411f-4e90-9570-96b20c9dfc44 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:04.495104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75eba61-411f-4e90-9570-96b20c9dfc44 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:04.496449 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 240/486] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:46:04.508519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2081d9-2774-4df8-a68f-69880b082cc5 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:04.509700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2081d9-2774-4df8-a68f-69880b082cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:04.510194 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2081d9-2774-4df8-a68f-69880b082cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:04.529885 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8b2081d9-2774-4df8-a68f-69880b082cc5 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:04.531358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8b2081d9-2774-4df8-a68f-69880b082cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:04.551228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:04.627862 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a798ff-12a5-48ac-947d-96c7b02f3127 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:04.659963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 241/487] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:46:04.681536 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6ba4c-0f10-4c58-80ab-8e67f1a2f808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:04.682491 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6ba4c-0f10-4c58-80ab-8e67f1a2f808 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:04.683001 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6ba4c-0f10-4c58-80ab-8e67f1a2f808 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:04.683480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6ba4c-0f10-4c58-80ab-8e67f1a2f808 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:04.684409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6ba4c-0f10-4c58-80ab-8e67f1a2f808 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:04.684925 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6ba4c-0f10-4c58-80ab-8e67f1a2f808 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:04.687084 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6ba4c-0f10-4c58-80ab-8e67f1a2f808 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:04.702285 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 247/488] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:46:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:04.707460 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2081d9-2774-4df8-a68f-69880b082cc5 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:04.755444 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6ba4c-0f10-4c58-80ab-8e67f1a2f808 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:04.848345 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2081d9-2774-4df8-a68f-69880b082cc5 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:04.870333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:04.895134 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec3900-9329-4940-bb9b-7ff9ba627eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:04.895770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec3900-9329-4940-bb9b-7ff9ba627eca None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:04.899433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec3900-9329-4940-bb9b-7ff9ba627eca None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:04.899433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec3900-9329-4940-bb9b-7ff9ba627eca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:04.901630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6ba4c-0f10-4c58-80ab-8e67f1a2f808 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:04.918695 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec3900-9329-4940-bb9b-7ff9ba627eca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:04.922374 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec3900-9329-4940-bb9b-7ff9ba627eca None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:04.935866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:04.949793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-1aec3900-9329-4940-bb9b-7ff9ba627eca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:04.949793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:04.949793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:04.949793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:04.949793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:04.949793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:04.949793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:04.949793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:04.949793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:04.949793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:04.949793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:04.949793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:04.949793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:04.949793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:04.949793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:04.949793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:04.950752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:04.950752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:04.950752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:04.950752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:04.950752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:04.950752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:04.950752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:04.950752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:04.950752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:04.950752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:04.950752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:04.950752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:04.950752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:04.950752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:04.950752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:04.950752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:04.950752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:04.951612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:04.951612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:04.951612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:04.951612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 242/489] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:04.958214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec3900-9329-4940-bb9b-7ff9ba627eca None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:04.970883 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e6e1b-6c8d-457b-a7cd-68b0f17c2260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:04.971518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e6e1b-6c8d-457b-a7cd-68b0f17c2260 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:04.971797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e6e1b-6c8d-457b-a7cd-68b0f17c2260 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:04.972127 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e6e1b-6c8d-457b-a7cd-68b0f17c2260 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:04.989356 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e6e1b-6c8d-457b-a7cd-68b0f17c2260 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:05.000082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-e24e6e1b-6c8d-457b-a7cd-68b0f17c2260 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:05.000082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:05.000082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:05.000082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:05.000082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:05.000082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:05.000082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:05.000082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:05.000082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:05.000082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:05.000082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:05.000082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:05.000082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:05.000082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:05.000082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:05.000082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:05.000976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:05.000976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:05.000976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:05.000976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:05.000976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:05.000976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:05.000976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:05.000976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:05.000976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:05.000976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:05.000976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:05.000976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:05.000976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:05.000976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:05.000976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:05.000976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:05.001786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:05.001786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:05.001786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:05.001786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:05.001786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:05.001786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:05.001786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:05.001786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:05.001786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:05.001786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:05.001786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:05.001786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:05.001786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 248/490] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:05.007587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e6e1b-6c8d-457b-a7cd-68b0f17c2260 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:05.105267 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec3900-9329-4940-bb9b-7ff9ba627eca None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:05.143252 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e6e1b-6c8d-457b-a7cd-68b0f17c2260 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:05.144041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c73bf-9ef1-44ac-b898-a89b45b48599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:05.144654 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c73bf-9ef1-44ac-b898-a89b45b48599 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:05.145115 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c73bf-9ef1-44ac-b898-a89b45b48599 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:05.145392 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c73bf-9ef1-44ac-b898-a89b45b48599 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:05.146892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c73bf-9ef1-44ac-b898-a89b45b48599 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:05.147206 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c73bf-9ef1-44ac-b898-a89b45b48599 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:05.148579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c73bf-9ef1-44ac-b898-a89b45b48599 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:05.168175 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 243/491] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:05.169167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c73bf-9ef1-44ac-b898-a89b45b48599 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:05.195714 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61cf25-e1d8-4ddb-b551-86f2563db8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:05.196629 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61cf25-e1d8-4ddb-b551-86f2563db8bf None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:05.196915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61cf25-e1d8-4ddb-b551-86f2563db8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:05.197195 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61cf25-e1d8-4ddb-b551-86f2563db8bf None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:05.198174 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61cf25-e1d8-4ddb-b551-86f2563db8bf None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:05.198506 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61cf25-e1d8-4ddb-b551-86f2563db8bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:05.201235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61cf25-e1d8-4ddb-b551-86f2563db8bf None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:05.202399 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:05.214549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 249/492] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:46:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:05.215682 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61cf25-e1d8-4ddb-b551-86f2563db8bf None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:05.269771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:05.286856 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c73bf-9ef1-44ac-b898-a89b45b48599 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:05.324600 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70190697-2618-4d2d-81e3-26335f153090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:05.325233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70190697-2618-4d2d-81e3-26335f153090 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:05.325614 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70190697-2618-4d2d-81e3-26335f153090 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:05.326464 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70190697-2618-4d2d-81e3-26335f153090 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:05.334127 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61cf25-e1d8-4ddb-b551-86f2563db8bf None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:05.337826 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70190697-2618-4d2d-81e3-26335f153090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:05.342659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-70190697-2618-4d2d-81e3-26335f153090 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:05.342659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:05.342659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:05.342659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:05.342659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:05.342659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:05.342659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:05.342659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:05.342659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:05.342659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:05.342659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:05.342659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:05.342659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:05.342659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:05.342659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:05.342659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:05.343627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:05.343627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:05.343627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:05.343627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:05.343627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:05.343627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:05.343627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:05.343627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:05.343627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:05.343627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:05.343627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:05.343627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:05.343627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:05.343627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:05.343627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:05.343627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:05.344415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:05.344415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:05.344415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:05.344415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:05.344415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:05.344415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:05.344415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:05.344415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:05.344415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:05.344415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:05.344415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:05.344415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:05.344415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 244/493] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:05.345108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70190697-2618-4d2d-81e3-26335f153090 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:05.374588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c30452-91e7-4af0-8969-382b6e8616cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:05.375226 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c30452-91e7-4af0-8969-382b6e8616cc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:05.375511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c30452-91e7-4af0-8969-382b6e8616cc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:05.375788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c30452-91e7-4af0-8969-382b6e8616cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:05.383776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c30452-91e7-4af0-8969-382b6e8616cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:05.385340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c30452-91e7-4af0-8969-382b6e8616cc None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:05.399480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-58c30452-91e7-4af0-8969-382b6e8616cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 11:46:05.399480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:05.399480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:05.399480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:05.399480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:05.399480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:05.399480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:05.399480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:05.399480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:05.399480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:05.399480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:05.399480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:05.399480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:05.399480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:05.399480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:05.399480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:05.400462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:05.400462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:05.400462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:05.400462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:05.400462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:05.400462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:05.400462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:05.400462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:05.400462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:05.400462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:05.400462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:05.400462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:05.400462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:05.400462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:05.400462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:05.400462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:05.400462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:05.400462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:05.401357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:05.401357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:05.401357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:05.401357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:05.401357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:05.401357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:05.401357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:05.401357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:05.401357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 11:46:05.401357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:05.401357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 250/494] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Sep 14 11:46:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:05.402420 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c214c82-b71b-43e3-8d9e-fcc959cb27c9 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:05.402420 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c214c82-b71b-43e3-8d9e-fcc959cb27c9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:05.402420 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c214c82-b71b-43e3-8d9e-fcc959cb27c9 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:05.403591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 251/495] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:05.404196 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c214c82-b71b-43e3-8d9e-fcc959cb27c9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:05.441477 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:05.456613 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70190697-2618-4d2d-81e3-26335f153090 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:05.491526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5391583-6202-4b2a-9ac3-495b8bf13b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:05.492153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5391583-6202-4b2a-9ac3-495b8bf13b4a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:05.492435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5391583-6202-4b2a-9ac3-495b8bf13b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:05.492710 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5391583-6202-4b2a-9ac3-495b8bf13b4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:05.505432 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5391583-6202-4b2a-9ac3-495b8bf13b4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:05.506946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5391583-6202-4b2a-9ac3-495b8bf13b4a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:05.517813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:05.522606 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c214c82-b71b-43e3-8d9e-fcc959cb27c9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:05.523319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d5391583-6202-4b2a-9ac3-495b8bf13b4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:05.523319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:05.523319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:05.523319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:05.523319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:05.523319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:05.523319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:05.523319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:05.523319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:05.523319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:05.523319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:05.523319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:05.523319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:05.523319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:05.523319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:05.523319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:05.524217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:05.524217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:05.524217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:05.524217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:05.524217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:05.524217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:05.524217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:05.524217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:05.524217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:05.524217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:05.524217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:05.524217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:05.524217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:05.524217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:05.524217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:05.524217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:05.524217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:05.525156 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:05.525156 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:05.525156 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:05.529128 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 245/496] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:05.530000 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5391583-6202-4b2a-9ac3-495b8bf13b4a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:05.559253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481cf71d-4338-456e-9cc5-0e626c40e7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:05.559859 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cf71d-4338-456e-9cc5-0e626c40e7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:05.560698 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cf71d-4338-456e-9cc5-0e626c40e7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:05.560976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cf71d-4338-456e-9cc5-0e626c40e7e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:05.561741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481cf71d-4338-456e-9cc5-0e626c40e7e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:05.562079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481cf71d-4338-456e-9cc5-0e626c40e7e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:05.563536 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481cf71d-4338-456e-9cc5-0e626c40e7e0 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:05.574186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 252/497] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:46:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:05.578266 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63594c-baef-4dc1-b699-637566980081 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:05.578646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63594c-baef-4dc1-b699-637566980081 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:05.578994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63594c-baef-4dc1-b699-637566980081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:05.599537 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-7a63594c-baef-4dc1-b699-637566980081 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:05.601143 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7a63594c-baef-4dc1-b699-637566980081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:05.641958 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5391583-6202-4b2a-9ac3-495b8bf13b4a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:05.685904 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 253/498] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:05.686094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd1100-b266-4b7d-b35f-df3233d0f2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:05.686219 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63594c-baef-4dc1-b699-637566980081 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:05.686319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd1100-b266-4b7d-b35f-df3233d0f2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:05.686319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd1100-b266-4b7d-b35f-df3233d0f2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:05.686319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd1100-b266-4b7d-b35f-df3233d0f2a0 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:05.686319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd1100-b266-4b7d-b35f-df3233d0f2a0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:05.686319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd1100-b266-4b7d-b35f-df3233d0f2a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:05.686319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd1100-b266-4b7d-b35f-df3233d0f2a0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:05.701274 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:05.721564 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 246/499] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Sep 14 11:46:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:05.722530 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd1100-b266-4b7d-b35f-df3233d0f2a0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:05.785644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:05.796025 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63594c-baef-4dc1-b699-637566980081 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:05.836176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11b81d-40ac-48ef-b219-116b7f249b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:05.836784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11b81d-40ac-48ef-b219-116b7f249b59 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:05.837078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11b81d-40ac-48ef-b219-116b7f249b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:05.837357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11b81d-40ac-48ef-b219-116b7f249b59 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:05.838043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11b81d-40ac-48ef-b219-116b7f249b59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:05.838353 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11b81d-40ac-48ef-b219-116b7f249b59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:05.839052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd1100-b266-4b7d-b35f-df3233d0f2a0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:05.839747 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11b81d-40ac-48ef-b219-116b7f249b59 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:05.851024 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 254/500] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:05.852207 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fa63d-e6dc-4387-bc76-245118d8c8ee None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:05.852496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fa63d-e6dc-4387-bc76-245118d8c8ee None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:05.852773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fa63d-e6dc-4387-bc76-245118d8c8ee None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:05.854151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 255/501] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:05.858644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fa63d-e6dc-4387-bc76-245118d8c8ee None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:05.883430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa738b-64c1-44fe-8e9e-4c9ab2e381de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:05.884064 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa738b-64c1-44fe-8e9e-4c9ab2e381de None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:05.884345 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa738b-64c1-44fe-8e9e-4c9ab2e381de None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:05.884618 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa738b-64c1-44fe-8e9e-4c9ab2e381de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:05.894641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa738b-64c1-44fe-8e9e-4c9ab2e381de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:05.896091 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa738b-64c1-44fe-8e9e-4c9ab2e381de None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:05.912336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-6eaa738b-64c1-44fe-8e9e-4c9ab2e381de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:05.912336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:05.912336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:05.912336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:05.912336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:05.912336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:05.912336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:05.912336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:05.912336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:05.912336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:05.912336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:05.912336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:05.912336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:05.912336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:05.912336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:05.912336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:05.913337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:05.913337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:05.913337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:05.913337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:05.913337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:05.913337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:05.913337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:05.913337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:05.913337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:05.913337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:05.913337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:05.913337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:05.913337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:05.913337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:05.913337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:05.913337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:05.913337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:05.914593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:05.914593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:05.914593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:05.917997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 247/502] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:05.918911 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa738b-64c1-44fe-8e9e-4c9ab2e381de None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:05.965505 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fa63d-e6dc-4387-bc76-245118d8c8ee None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:05.970343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:05.998915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57160822-f9f1-4137-8299-d3c6c8855f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:05.999516 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57160822-f9f1-4137-8299-d3c6c8855f47 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:05.999801 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57160822-f9f1-4137-8299-d3c6c8855f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:06.000079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57160822-f9f1-4137-8299-d3c6c8855f47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:06.018538 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:06.021207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa738b-64c1-44fe-8e9e-4c9ab2e381de None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:06.055334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7379214-59a0-43d9-97a7-7624e8d5c317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:06.056190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7379214-59a0-43d9-97a7-7624e8d5c317 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:06.056489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7379214-59a0-43d9-97a7-7624e8d5c317 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:06.056773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7379214-59a0-43d9-97a7-7624e8d5c317 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:06.064056 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7379214-59a0-43d9-97a7-7624e8d5c317 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:06.065498 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7379214-59a0-43d9-97a7-7624e8d5c317 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:06.067908 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57160822-f9f1-4137-8299-d3c6c8855f47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:06.069419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57160822-f9f1-4137-8299-d3c6c8855f47 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:06.077797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-e7379214-59a0-43d9-97a7-7624e8d5c317 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 14 11:46:06.077797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:06.077797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:06.077797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:06.077797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:06.077797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:06.077797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:06.077797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:06.077797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:06.077797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:06.077797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:06.077797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:06.077797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:06.077797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:06.077797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:06.077797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:06.078707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:06.078707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:06.078707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:06.078707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:06.078707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:06.078707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:06.078707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:06.078707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:06.078707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:06.078707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:06.078707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:06.078707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:06.078707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:06.078707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:06.078707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:06.078707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:06.078707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:06.078707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:06.079769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:06.079769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:06.079769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:06.079769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:06.079769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:06.079769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:06.079769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:06.079769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:06.079769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 14 11:46:06.079769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:06.079769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 248/503] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:46:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:06.080567 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746298c-7f63-4402-a18d-34f134dc9e70 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:06.080567 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746298c-7f63-4402-a18d-34f134dc9e70 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:06.080567 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746298c-7f63-4402-a18d-34f134dc9e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:06.098707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6746298c-7f63-4402-a18d-34f134dc9e70 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:06.099991 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6746298c-7f63-4402-a18d-34f134dc9e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:06.166315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 249/504] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:06.167148 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746298c-7f63-4402-a18d-34f134dc9e70 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:06.173416 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 256/505] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:46:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:46:06.174285 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57160822-f9f1-4137-8299-d3c6c8855f47 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:06.215108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:06.224588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:06.273649 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57160822-f9f1-4137-8299-d3c6c8855f47 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:06.277677 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746298c-7f63-4402-a18d-34f134dc9e70 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:06.311189 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544e6d6-e4b3-4419-912e-7d958e4300ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:06.311749 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544e6d6-e4b3-4419-912e-7d958e4300ce None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:06.312034 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544e6d6-e4b3-4419-912e-7d958e4300ce None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:06.312307 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544e6d6-e4b3-4419-912e-7d958e4300ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:06.313415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f39e5-af89-42d5-9c86-fc1dced5ec99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:06.314051 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f39e5-af89-42d5-9c86-fc1dced5ec99 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:06.314334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f39e5-af89-42d5-9c86-fc1dced5ec99 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:06.314608 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f39e5-af89-42d5-9c86-fc1dced5ec99 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:06.315242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f39e5-af89-42d5-9c86-fc1dced5ec99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:06.315549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f39e5-af89-42d5-9c86-fc1dced5ec99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:06.316916 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f39e5-af89-42d5-9c86-fc1dced5ec99 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:06.322856 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544e6d6-e4b3-4419-912e-7d958e4300ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:06.328208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-b544e6d6-e4b3-4419-912e-7d958e4300ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:06.328208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:06.328208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:06.328208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:06.328208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:06.328208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:06.328208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:06.328208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:06.328208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:06.328208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:06.328208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:06.328208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:06.328208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:06.328208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:06.328208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:06.328208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:06.329106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:06.329106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:06.329106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:06.329106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:06.329106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:06.329106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:06.329106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:06.329106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:06.329106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:06.329106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:06.329106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:06.329106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:06.329106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:06.329106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:06.329106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:06.329106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:06.329945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:06.329945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:06.329945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:06.329945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:06.329945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:06.329945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:06.329945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:06.329945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:06.329945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:06.329945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:06.329945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:06.329945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:06.330558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 250/506] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:06.334003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 257/507] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:06.334412 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f39e5-af89-42d5-9c86-fc1dced5ec99 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:06.335227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544e6d6-e4b3-4419-912e-7d958e4300ce None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:06.438440 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:06.442180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f39e5-af89-42d5-9c86-fc1dced5ec99 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:06.444821 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544e6d6-e4b3-4419-912e-7d958e4300ce None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:06.455685 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:06.476717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26335f32-980e-4017-bb4e-3cef48fdfb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:06.477324 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26335f32-980e-4017-bb4e-3cef48fdfb33 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:06.477612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26335f32-980e-4017-bb4e-3cef48fdfb33 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:06.477913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26335f32-980e-4017-bb4e-3cef48fdfb33 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:06.478642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26335f32-980e-4017-bb4e-3cef48fdfb33 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:06.478955 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26335f32-980e-4017-bb4e-3cef48fdfb33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:06.479635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcaa85c-a939-41d2-9ab7-fd70c434a706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:06.481187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcaa85c-a939-41d2-9ab7-fd70c434a706 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:06.481292 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26335f32-980e-4017-bb4e-3cef48fdfb33 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:06.481377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcaa85c-a939-41d2-9ab7-fd70c434a706 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:06.481377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcaa85c-a939-41d2-9ab7-fd70c434a706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:06.491477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcaa85c-a939-41d2-9ab7-fd70c434a706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:06.492984 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcaa85c-a939-41d2-9ab7-fd70c434a706 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:06.497814 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 251/508] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Sep 14 11:46:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:06.498642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26335f32-980e-4017-bb4e-3cef48fdfb33 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:06.507833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-dbcaa85c-a939-41d2-9ab7-fd70c434a706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:06.507833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:06.507833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:06.507833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:06.507833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:06.507833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:06.507833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:06.507833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:06.507833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:06.507833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:06.507833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:06.507833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:06.507833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:06.507833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:06.507833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:06.507833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:06.508920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:06.508920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:06.508920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:06.508920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:06.508920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:06.508920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:06.508920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:06.508920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:06.508920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:06.508920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:06.508920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:06.508920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:06.508920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:06.508920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:06.508920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:06.508920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:06.508920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:06.509790 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:06.509790 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:06.509790 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:06.509790 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 258/509] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:06.510347 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcaa85c-a939-41d2-9ab7-fd70c434a706 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:06.601773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26335f32-980e-4017-bb4e-3cef48fdfb33 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:06.610570 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcaa85c-a939-41d2-9ab7-fd70c434a706 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:06.636402 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa79d7-1f75-41cd-9e56-03dac97cabb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:06.637032 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa79d7-1f75-41cd-9e56-03dac97cabb0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:06.637314 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa79d7-1f75-41cd-9e56-03dac97cabb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:06.637590 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa79d7-1f75-41cd-9e56-03dac97cabb0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:06.643767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ef3ed-e405-4129-9d0a-fb35f1629147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:06.644363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ef3ed-e405-4129-9d0a-fb35f1629147 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:06.644648 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ef3ed-e405-4129-9d0a-fb35f1629147 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:06.644925 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ef3ed-e405-4129-9d0a-fb35f1629147 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:06.645010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa79d7-1f75-41cd-9e56-03dac97cabb0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:06.645587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ef3ed-e405-4129-9d0a-fb35f1629147 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:06.645920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ef3ed-e405-4129-9d0a-fb35f1629147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:06.646448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa79d7-1f75-41cd-9e56-03dac97cabb0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:06.647304 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ef3ed-e405-4129-9d0a-fb35f1629147 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:06.655662 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 259/510] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:46:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:06.656558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ef3ed-e405-4129-9d0a-fb35f1629147 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:06.658388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-a1aa79d7-1f75-41cd-9e56-03dac97cabb0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 11:46:06.658388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:06.658388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:06.658388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:06.658388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:06.658388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:06.658388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:06.658388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:06.658388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:06.658388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:06.658388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:06.658388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:06.658388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:06.658388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:06.658388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:06.658388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:06.659270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:06.659270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:06.659270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:06.659270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:06.659270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:06.659270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:06.659270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:06.659270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:06.659270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:06.659270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:06.659270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:06.659270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:06.659270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:06.659270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:06.659270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:06.659270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:06.659270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:06.659270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:06.660169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:06.660169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:06.660169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:06.660169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:06.660169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:06.660169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:06.660169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:06.660169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:06.660169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 11:46:06.660169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:06.660169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 252/511] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:46:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:06.660741 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa79d7-1f75-41cd-9e56-03dac97cabb0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:06.678745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:06.684330 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:06.757918 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ef3ed-e405-4129-9d0a-fb35f1629147 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:06.758341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa79d7-1f75-41cd-9e56-03dac97cabb0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:06.791372 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3487e04-388c-4171-8f26-1a340d878e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:06.791997 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3487e04-388c-4171-8f26-1a340d878e04 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:06.792129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc70cf-1799-4241-a7bf-94a69af0e07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:06.792280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3487e04-388c-4171-8f26-1a340d878e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:06.792554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3487e04-388c-4171-8f26-1a340d878e04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:06.792755 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc70cf-1799-4241-a7bf-94a69af0e07f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:06.793048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc70cf-1799-4241-a7bf-94a69af0e07f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:06.793324 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc70cf-1799-4241-a7bf-94a69af0e07f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:06.803111 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3487e04-388c-4171-8f26-1a340d878e04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:06.804567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3487e04-388c-4171-8f26-1a340d878e04 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:06.820993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-e3487e04-388c-4171-8f26-1a340d878e04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:06.820993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:06.820993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:06.820993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:06.820993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:06.820993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:06.820993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:06.820993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:06.820993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:06.820993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:06.820993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:06.820993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:06.820993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:06.820993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:06.820993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:06.820993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:06.822340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:06.822340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:06.822340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:06.822340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:06.822340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:06.822340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:06.822340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:06.822340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:06.822340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:06.822340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:06.822340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:06.822340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:06.822340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:06.822340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:06.822340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:06.822340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:06.822340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:06.823184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:06.823184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:06.823184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:06.823184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 260/512] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:06.823184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3487e04-388c-4171-8f26-1a340d878e04 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:06.862539 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc70cf-1799-4241-a7bf-94a69af0e07f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:06.863936 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc70cf-1799-4241-a7bf-94a69af0e07f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:06.881207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:06.897383 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:06.920074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3487e04-388c-4171-8f26-1a340d878e04 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:06.954688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bc110-9f77-4826-9b46-45c71ea92847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:06.955288 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bc110-9f77-4826-9b46-45c71ea92847 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:06.955572 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bc110-9f77-4826-9b46-45c71ea92847 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:06.955847 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bc110-9f77-4826-9b46-45c71ea92847 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:06.956487 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bc110-9f77-4826-9b46-45c71ea92847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:06.956794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bc110-9f77-4826-9b46-45c71ea92847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:06.958192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bc110-9f77-4826-9b46-45c71ea92847 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:06.962042 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 253/513] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:46:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:46:06.962879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc70cf-1799-4241-a7bf-94a69af0e07f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:06.972094 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 261/514] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:06.972912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bc110-9f77-4826-9b46-45c71ea92847 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:07.063114 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc70cf-1799-4241-a7bf-94a69af0e07f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:07.069961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bc110-9f77-4826-9b46-45c71ea92847 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:07.078984 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:07.095897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3ee8d-cb40-4790-bb3c-038e1a347a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:07.096552 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3ee8d-cb40-4790-bb3c-038e1a347a13 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:07.096835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3ee8d-cb40-4790-bb3c-038e1a347a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:07.097105 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3ee8d-cb40-4790-bb3c-038e1a347a13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:07.104328 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61677f13-f9df-4673-9396-05b43f80e0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:07.104982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61677f13-f9df-4673-9396-05b43f80e0f3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:07.105306 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61677f13-f9df-4673-9396-05b43f80e0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:07.105596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61677f13-f9df-4673-9396-05b43f80e0f3 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:07.106367 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61677f13-f9df-4673-9396-05b43f80e0f3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:07.106681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61677f13-f9df-4673-9396-05b43f80e0f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:07.107944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b3ee8d-cb40-4790-bb3c-038e1a347a13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:07.108637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61677f13-f9df-4673-9396-05b43f80e0f3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:07.109377 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b3ee8d-cb40-4790-bb3c-038e1a347a13 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:07.119456 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:07.124229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-79b3ee8d-cb40-4790-bb3c-038e1a347a13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:07.124229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:07.124229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:07.124229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:07.124229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:07.124229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:07.124229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:07.124229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:07.124229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:07.124229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:07.124229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:07.124229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:07.124229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:07.124229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:07.124229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:07.124229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:07.125242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:07.125242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:07.125242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:07.125242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:07.125242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:07.125242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:07.125242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:07.125242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:07.125242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:07.125242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:07.125242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:07.125242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:07.125242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:07.125242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:07.125242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:07.125242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:07.125242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:07.128514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:07.128514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:07.128514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:07.128514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 254/515] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:07.128514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3ee8d-cb40-4790-bb3c-038e1a347a13 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:07.148135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 262/516] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Sep 14 11:46:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:07.149138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61677f13-f9df-4673-9396-05b43f80e0f3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:07.229728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3ee8d-cb40-4790-bb3c-038e1a347a13 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:07.246551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61677f13-f9df-4673-9396-05b43f80e0f3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:07.263625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae671f-d05d-4e8d-8a8c-32f65c3d3894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:07.264419 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae671f-d05d-4e8d-8a8c-32f65c3d3894 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:07.264700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae671f-d05d-4e8d-8a8c-32f65c3d3894 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:07.264977 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae671f-d05d-4e8d-8a8c-32f65c3d3894 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:07.265611 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae671f-d05d-4e8d-8a8c-32f65c3d3894 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:07.266008 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae671f-d05d-4e8d-8a8c-32f65c3d3894 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:07.267401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae671f-d05d-4e8d-8a8c-32f65c3d3894 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:07.280591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 255/517] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:07.280591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae671f-d05d-4e8d-8a8c-32f65c3d3894 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:07.281227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83236538-b5b8-494c-bd88-015df368ff1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:07.281883 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83236538-b5b8-494c-bd88-015df368ff1d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:07.282167 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83236538-b5b8-494c-bd88-015df368ff1d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:07.282441 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83236538-b5b8-494c-bd88-015df368ff1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:07.294137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83236538-b5b8-494c-bd88-015df368ff1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:07.296123 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83236538-b5b8-494c-bd88-015df368ff1d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:07.307561 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:07.311612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-83236538-b5b8-494c-bd88-015df368ff1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:07.311612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:07.311612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:07.311612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:07.311612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:07.311612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:07.311612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:07.311612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:07.311612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:07.311612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:07.311612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:07.311612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:07.311612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:07.311612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:07.311612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:07.311612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:07.312575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:07.312575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:07.312575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:07.312575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:07.312575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:07.312575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:07.312575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:07.312575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:07.312575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:07.312575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:07.312575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:07.312575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:07.312575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:07.312575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:07.312575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:07.312575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:07.312575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:07.313520 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:07.313520 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:07.313520 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:07.317222 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 263/518] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:07.318023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83236538-b5b8-494c-bd88-015df368ff1d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:07.364250 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:07.379291 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae671f-d05d-4e8d-8a8c-32f65c3d3894 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:07.412606 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41a1f6-71f0-4ce0-8014-2d6b55d5af6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:07.413236 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41a1f6-71f0-4ce0-8014-2d6b55d5af6a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:07.413519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41a1f6-71f0-4ce0-8014-2d6b55d5af6a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:07.413796 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41a1f6-71f0-4ce0-8014-2d6b55d5af6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:07.415719 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83236538-b5b8-494c-bd88-015df368ff1d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:07.420805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41a1f6-71f0-4ce0-8014-2d6b55d5af6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:07.422268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41a1f6-71f0-4ce0-8014-2d6b55d5af6a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:07.434942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-3c41a1f6-71f0-4ce0-8014-2d6b55d5af6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:07.434942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:07.434942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:07.434942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:07.434942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:07.434942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:07.434942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:07.434942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:07.434942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:07.434942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:07.434942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:07.434942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:07.434942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:07.434942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:07.434942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:07.436321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:07.436321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:07.436321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:07.436321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:07.436321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:07.436321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:07.436321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:07.436321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:07.436321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:07.436321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:07.436321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:07.436321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:07.436321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:07.436321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:07.436321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:07.436321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:07.436321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:07.437170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:07.437170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:07.437170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:07.437170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:07.437170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:07.437170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:07.437170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:07.437170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:07.437170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 256/519] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:07.437651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41a1f6-71f0-4ce0-8014-2d6b55d5af6a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:07.452317 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c1591-29c4-48ef-b01f-9c9cdbca33c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:07.452923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c1591-29c4-48ef-b01f-9c9cdbca33c5 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:07.453210 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c1591-29c4-48ef-b01f-9c9cdbca33c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:07.453492 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c1591-29c4-48ef-b01f-9c9cdbca33c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:07.454177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c1591-29c4-48ef-b01f-9c9cdbca33c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:07.454487 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c1591-29c4-48ef-b01f-9c9cdbca33c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:07.455873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c1591-29c4-48ef-b01f-9c9cdbca33c5 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:07.468012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 264/520] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:07.468625 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c1591-29c4-48ef-b01f-9c9cdbca33c5 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:07.526029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:07.538081 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41a1f6-71f0-4ce0-8014-2d6b55d5af6a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:07.568245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c1591-29c4-48ef-b01f-9c9cdbca33c5 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:07.572179 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f20c7-5be8-4c67-8503-e1c686b9ea32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:07.572846 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f20c7-5be8-4c67-8503-e1c686b9ea32 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:07.573125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f20c7-5be8-4c67-8503-e1c686b9ea32 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:07.573397 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f20c7-5be8-4c67-8503-e1c686b9ea32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:07.575415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:07.602684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2a838-2092-4960-b994-8ed5736b3767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:07.603287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2a838-2092-4960-b994-8ed5736b3767 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:07.603573 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2a838-2092-4960-b994-8ed5736b3767 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:07.603849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2a838-2092-4960-b994-8ed5736b3767 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:07.604497 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2a838-2092-4960-b994-8ed5736b3767 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:07.606017 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2a838-2092-4960-b994-8ed5736b3767 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:07.639214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f20c7-5be8-4c67-8503-e1c686b9ea32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:07.640576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f20c7-5be8-4c67-8503-e1c686b9ea32 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:07.689024 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 265/521] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 14 11:46:07 2022] POST /identity/v3/users => generated 247 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:46:07.689943 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2a838-2092-4960-b994-8ed5736b3767 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:07.738496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:07.758113 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 257/522] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:46:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:46:07.758113 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f20c7-5be8-4c67-8503-e1c686b9ea32 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:07.819431 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2a838-2092-4960-b994-8ed5736b3767 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:07.862963 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d23d1-5d98-4a22-b2db-538354eb9a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:07.863577 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d23d1-5d98-4a22-b2db-538354eb9a70 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:07.863856 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d23d1-5d98-4a22-b2db-538354eb9a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:07.864132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d23d1-5d98-4a22-b2db-538354eb9a70 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:07.864777 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d23d1-5d98-4a22-b2db-538354eb9a70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:07.865084 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d23d1-5d98-4a22-b2db-538354eb9a70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:07.866505 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d23d1-5d98-4a22-b2db-538354eb9a70 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:07.870967 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f20c7-5be8-4c67-8503-e1c686b9ea32 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:07.882551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 266/523] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:07.882551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d23d1-5d98-4a22-b2db-538354eb9a70 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:07.907236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:07.918289 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e59e8-0bdc-4e73-809e-e0b066c7f644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:07.918965 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e59e8-0bdc-4e73-809e-e0b066c7f644 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:07.919173 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e59e8-0bdc-4e73-809e-e0b066c7f644 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:07.919476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e59e8-0bdc-4e73-809e-e0b066c7f644 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:07.920161 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4e59e8-0bdc-4e73-809e-e0b066c7f644 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:07.920462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4e59e8-0bdc-4e73-809e-e0b066c7f644 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:07.921928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4e59e8-0bdc-4e73-809e-e0b066c7f644 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:07.946081 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 258/524] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:07.947813 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e59e8-0bdc-4e73-809e-e0b066c7f644 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:07.989293 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d23d1-5d98-4a22-b2db-538354eb9a70 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:08.003725 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:08.026345 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d1d8d-1fd6-4216-8618-5538499dfe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:08.027005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d1d8d-1fd6-4216-8618-5538499dfe62 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:08.027286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d1d8d-1fd6-4216-8618-5538499dfe62 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:08.027559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d1d8d-1fd6-4216-8618-5538499dfe62 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:08.035457 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d1d8d-1fd6-4216-8618-5538499dfe62 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:08.036908 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d1d8d-1fd6-4216-8618-5538499dfe62 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:08.048738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-7e8d1d8d-1fd6-4216-8618-5538499dfe62 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 11:46:08.048738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:08.048738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:08.048738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:08.048738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:08.048738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:08.048738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:08.048738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:08.048738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:08.048738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:08.048738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:08.048738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:08.048738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:08.048738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:08.048738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:08.048738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:08.050162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:08.050162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:08.050162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:08.050162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:08.050162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:08.050162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:08.050162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:08.050162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:08.050162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:08.050162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:08.050162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:08.050162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:08.050162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:08.050162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:08.050162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:08.050162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:08.050162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:08.050162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:08.051096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:08.051096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:08.051096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:08.051096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:08.051096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:08.051096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:08.051096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:08.051096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:08.051096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 11:46:08.051096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:08.051096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 267/525] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:08.051096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d1d8d-1fd6-4216-8618-5538499dfe62 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:08.067794 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e59e8-0bdc-4e73-809e-e0b066c7f644 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:08.103838 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d397d1-fec9-4ad5-9059-8610c1f44b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:08.104501 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d397d1-fec9-4ad5-9059-8610c1f44b81 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:08.104783 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d397d1-fec9-4ad5-9059-8610c1f44b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:08.105063 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d397d1-fec9-4ad5-9059-8610c1f44b81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:08.115773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d397d1-fec9-4ad5-9059-8610c1f44b81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:08.117251 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d397d1-fec9-4ad5-9059-8610c1f44b81 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:08.133414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-26d397d1-fec9-4ad5-9059-8610c1f44b81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:08.133414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:08.133414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:08.133414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:08.133414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:08.133414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:08.133414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:08.133414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:08.133414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:08.133414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:08.133414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:08.133414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:08.133414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:08.133414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:08.133414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:08.133414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:08.134682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:08.134682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:08.134682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:08.134682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:08.134682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:08.134682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:08.134682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:08.134682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:08.134682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:08.134682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:08.134682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:08.134682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:08.134682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:08.134682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:08.134682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:08.134682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:08.134682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:08.135531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:08.135531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:08.135531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:08.135531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 259/526] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:08.136116 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d397d1-fec9-4ad5-9059-8610c1f44b81 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:08.139896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:08.158807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d1d8d-1fd6-4216-8618-5538499dfe62 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:08.192738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc50123-ae12-4842-a119-a6870e1b59ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:08.193363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc50123-ae12-4842-a119-a6870e1b59ab None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:08.193647 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc50123-ae12-4842-a119-a6870e1b59ab None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:08.193954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc50123-ae12-4842-a119-a6870e1b59ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:08.201130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc50123-ae12-4842-a119-a6870e1b59ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:08.203411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc50123-ae12-4842-a119-a6870e1b59ab None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:08.214769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-5bc50123-ae12-4842-a119-a6870e1b59ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:08.214769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:08.214769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:08.214769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:08.214769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:08.214769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:08.214769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:08.214769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:08.214769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:08.214769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:08.214769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:08.214769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:08.214769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:08.214769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:08.214769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:08.215628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:08.215628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:08.215628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:08.215628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:08.215628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:08.215628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:08.215628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:08.215628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:08.215628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:08.215628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:08.215628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:08.215628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:08.215628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:08.215628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:08.215628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:08.215628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:08.215628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:08.221779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:08.221779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:08.221779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:08.221779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:08.221779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:08.221779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:08.221779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:08.221779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:08.221779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 268/527] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:08.221779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc50123-ae12-4842-a119-a6870e1b59ab None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:08.248788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:08.251720 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d397d1-fec9-4ad5-9059-8610c1f44b81 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:08.290089 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fc5a7-af82-4fbb-8fd3-c4792345cf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:08.290701 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fc5a7-af82-4fbb-8fd3-c4792345cf9b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:08.290986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fc5a7-af82-4fbb-8fd3-c4792345cf9b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:08.291269 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fc5a7-af82-4fbb-8fd3-c4792345cf9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:08.291997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5fc5a7-af82-4fbb-8fd3-c4792345cf9b None admin] RBAC: Adding query filter params (role.id=d1bb55f25dff4694b8675e0ef5fc4ec7, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=38b093bc4c1d4ebcbd2a6bb16b70c029) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:08.292369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5fc5a7-af82-4fbb-8fd3-c4792345cf9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:08.293816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5fc5a7-af82-4fbb-8fd3-c4792345cf9b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:08.299819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 260/528] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:46:08 2022] GET /identity/v3/role_assignments?user.id=38b093bc4c1d4ebcbd2a6bb16b70c029&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:08.300617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fc5a7-af82-4fbb-8fd3-c4792345cf9b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:08.349744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc50123-ae12-4842-a119-a6870e1b59ab None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:08.392473 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f6242-4c94-4608-8d2c-eda3342c1be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:08.393174 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f6242-4c94-4608-8d2c-eda3342c1be8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:08.393489 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f6242-4c94-4608-8d2c-eda3342c1be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:08.393842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f6242-4c94-4608-8d2c-eda3342c1be8 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:08.394549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f6242-4c94-4608-8d2c-eda3342c1be8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:08.394861 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f6242-4c94-4608-8d2c-eda3342c1be8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:08.396402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f6242-4c94-4608-8d2c-eda3342c1be8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:08.413610 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:08.423636 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 269/529] 10.222.0.62 () {56 vars in 1139 bytes} [Wed Sep 14 11:46:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:08.424886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f6242-4c94-4608-8d2c-eda3342c1be8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:08.437329 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fc5a7-af82-4fbb-8fd3-c4792345cf9b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:08.480944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8544d-162c-4eb9-99de-aaf622937ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:08.481663 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8544d-162c-4eb9-99de-aaf622937ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:08.482542 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8544d-162c-4eb9-99de-aaf622937ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:08.482823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8544d-162c-4eb9-99de-aaf622937ce6 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:08.483475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8544d-162c-4eb9-99de-aaf622937ce6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:08.483782 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8544d-162c-4eb9-99de-aaf622937ce6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:08.485334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8544d-162c-4eb9-99de-aaf622937ce6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:08.498785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 261/530] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:08.499641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8544d-162c-4eb9-99de-aaf622937ce6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:08.529279 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:08.548070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f6242-4c94-4608-8d2c-eda3342c1be8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:08.600299 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee18b3-8869-4b10-9aac-b87d01eb4c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:08.600937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee18b3-8869-4b10-9aac-b87d01eb4c8d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:08.601216 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee18b3-8869-4b10-9aac-b87d01eb4c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:08.601493 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee18b3-8869-4b10-9aac-b87d01eb4c8d None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:08.602247 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee18b3-8869-4b10-9aac-b87d01eb4c8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:08.602559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee18b3-8869-4b10-9aac-b87d01eb4c8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:08.604169 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee18b3-8869-4b10-9aac-b87d01eb4c8d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:08.619151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8544d-162c-4eb9-99de-aaf622937ce6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:08.633939 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 270/531] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:08.635140 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee18b3-8869-4b10-9aac-b87d01eb4c8d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:08.666791 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9904013-95dc-4121-9288-fcd1bb4f3a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:08.667888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9904013-95dc-4121-9288-fcd1bb4f3a87 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:08.668322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9904013-95dc-4121-9288-fcd1bb4f3a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:08.668804 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9904013-95dc-4121-9288-fcd1bb4f3a87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:08.714094 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:08.749720 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9904013-95dc-4121-9288-fcd1bb4f3a87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:08.751477 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9904013-95dc-4121-9288-fcd1bb4f3a87 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:08.767604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee18b3-8869-4b10-9aac-b87d01eb4c8d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:08.798307 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:08.814020 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475dfcf-f6bc-40e1-8220-25224fa9a617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:08.814879 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475dfcf-f6bc-40e1-8220-25224fa9a617 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:08.815311 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475dfcf-f6bc-40e1-8220-25224fa9a617 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:08.815746 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475dfcf-f6bc-40e1-8220-25224fa9a617 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:08.826670 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475dfcf-f6bc-40e1-8220-25224fa9a617 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:08.828757 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475dfcf-f6bc-40e1-8220-25224fa9a617 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:08.843903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-3475dfcf-f6bc-40e1-8220-25224fa9a617 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:08.843903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:08.843903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:08.843903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:08.843903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:08.843903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:08.843903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:08.843903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:08.843903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:08.843903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:08.843903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:08.843903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:08.843903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:08.843903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:08.843903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:08.844819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:08.844819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:08.844819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:08.844819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:08.844819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:08.844819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:08.844819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:08.844819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:08.844819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:08.844819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:08.844819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:08.844819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:08.844819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:08.844819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:08.844819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:08.844819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:08.844819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:08.845695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:08.845695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:08.845695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:08.845695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:08.845695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:08.845695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:08.845695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:08.845695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:08.848142 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 271/532] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:08.849150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475dfcf-f6bc-40e1-8220-25224fa9a617 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:08.882355 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 262/533] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:46:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:46:08.887120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9904013-95dc-4121-9288-fcd1bb4f3a87 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:08.966761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475dfcf-f6bc-40e1-8220-25224fa9a617 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:08.988588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:09.003774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08feb1b3-2daa-453a-9a6d-6e9521a6e906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:09.004023 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9904013-95dc-4121-9288-fcd1bb4f3a87 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:09.010961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08feb1b3-2daa-453a-9a6d-6e9521a6e906 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:09.010961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08feb1b3-2daa-453a-9a6d-6e9521a6e906 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:09.010961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08feb1b3-2daa-453a-9a6d-6e9521a6e906 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:09.010961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08feb1b3-2daa-453a-9a6d-6e9521a6e906 None admin] RBAC: Adding query filter params (role.id=d1bb55f25dff4694b8675e0ef5fc4ec7, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=4b05d0e65e9645f3810a6d96c8fed9fc) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:09.010961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08feb1b3-2daa-453a-9a6d-6e9521a6e906 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:09.010961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08feb1b3-2daa-453a-9a6d-6e9521a6e906 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:09.021890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 272/534] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:46:08 2022] GET /identity/v3/role_assignments?user.id=4b05d0e65e9645f3810a6d96c8fed9fc&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:09.021890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08feb1b3-2daa-453a-9a6d-6e9521a6e906 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:09.065843 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7414164-f9f1-4d11-a5d5-c7e5d9a6e863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:09.066702 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7414164-f9f1-4d11-a5d5-c7e5d9a6e863 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:09.067088 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7414164-f9f1-4d11-a5d5-c7e5d9a6e863 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:09.067469 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7414164-f9f1-4d11-a5d5-c7e5d9a6e863 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:09.068276 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7414164-f9f1-4d11-a5d5-c7e5d9a6e863 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:09.068692 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7414164-f9f1-4d11-a5d5-c7e5d9a6e863 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:09.070468 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7414164-f9f1-4d11-a5d5-c7e5d9a6e863 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:09.078261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:09.084643 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 263/535] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:09.090935 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7414164-f9f1-4d11-a5d5-c7e5d9a6e863 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:09.242539 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08feb1b3-2daa-453a-9a6d-6e9521a6e906 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:09.280992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7414164-f9f1-4d11-a5d5-c7e5d9a6e863 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:09.295429 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf49628-4d9e-4bd5-a697-8c5db86ef025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:09.296208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf49628-4d9e-4bd5-a697-8c5db86ef025 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:09.296595 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf49628-4d9e-4bd5-a697-8c5db86ef025 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:09.296973 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf49628-4d9e-4bd5-a697-8c5db86ef025 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:09.316221 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf49628-4d9e-4bd5-a697-8c5db86ef025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:09.320653 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf49628-4d9e-4bd5-a697-8c5db86ef025 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:09.344151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-faf49628-4d9e-4bd5-a697-8c5db86ef025 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:09.344151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:09.344151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:09.344151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:09.344151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:09.344151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:09.344151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:09.344151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:09.344151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:09.344151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:09.344151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:09.344151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:09.344151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:09.344151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:09.344151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:09.344151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:09.345287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:09.345287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:09.345287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:09.345287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:09.345287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:09.345287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:09.345287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:09.345287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:09.345287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:09.345287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:09.345287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:09.345287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:09.345287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:09.345287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:09.345287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:09.345287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:09.345287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:09.346399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:09.346399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:09.346399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:09.351464 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc3104-1b33-4aaa-ac85-d471ba14867e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:09.352191 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc3104-1b33-4aaa-ac85-d471ba14867e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:09.352578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc3104-1b33-4aaa-ac85-d471ba14867e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:09.352956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc3104-1b33-4aaa-ac85-d471ba14867e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:09.353796 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc3104-1b33-4aaa-ac85-d471ba14867e None admin] RBAC: Adding query filter params (role.id=d1bb55f25dff4694b8675e0ef5fc4ec7, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=d27c7ff726e443e89da94f282b5cddab) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:09.354230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc3104-1b33-4aaa-ac85-d471ba14867e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:09.355852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc3104-1b33-4aaa-ac85-d471ba14867e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:09.358778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 273/536] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:09.365847 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf49628-4d9e-4bd5-a697-8c5db86ef025 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:09.376876 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 264/537] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:46:09 2022] GET /identity/v3/role_assignments?user.id=d27c7ff726e443e89da94f282b5cddab&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:46:09.422246 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:09.505903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf49628-4d9e-4bd5-a697-8c5db86ef025 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:09.550070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5bcb08-0fde-4b2b-a81b-d1e9ab1df02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:09.550885 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5bcb08-0fde-4b2b-a81b-d1e9ab1df02d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:09.551270 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5bcb08-0fde-4b2b-a81b-d1e9ab1df02d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:09.551649 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5bcb08-0fde-4b2b-a81b-d1e9ab1df02d None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:09.552414 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5bcb08-0fde-4b2b-a81b-d1e9ab1df02d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:09.552823 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5bcb08-0fde-4b2b-a81b-d1e9ab1df02d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:09.554461 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5bcb08-0fde-4b2b-a81b-d1e9ab1df02d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:09.569230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 274/538] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:09.574894 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc3104-1b33-4aaa-ac85-d471ba14867e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:09.640905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:09.701958 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc3104-1b33-4aaa-ac85-d471ba14867e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:09.738289 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39521f-a151-47af-ab8c-bd4e9baceca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:09.739071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39521f-a151-47af-ab8c-bd4e9baceca3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:09.739452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39521f-a151-47af-ab8c-bd4e9baceca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:09.739825 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39521f-a151-47af-ab8c-bd4e9baceca3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:09.746917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39521f-a151-47af-ab8c-bd4e9baceca3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:09.748985 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39521f-a151-47af-ab8c-bd4e9baceca3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:09.760798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-0a39521f-a151-47af-ab8c-bd4e9baceca3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:09.760798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:09.760798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:09.760798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:09.760798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:09.760798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:09.760798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:09.760798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:09.760798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:09.760798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:09.760798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:09.760798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:09.760798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:09.760798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:09.760798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:09.761809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:09.761809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:09.761809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:09.761809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:09.761809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:09.761809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:09.761809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:09.761809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:09.761809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:09.761809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:09.761809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:09.761809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:09.761809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:09.761809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:09.761809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:09.761809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:09.761809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:09.762986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:09.762986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:09.762986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:09.762986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:09.762986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:09.762986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:09.762986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:09.762986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:09.769444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 265/539] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:09.770694 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5bcb08-0fde-4b2b-a81b-d1e9ab1df02d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:09.906865 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:09.913828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5bcb08-0fde-4b2b-a81b-d1e9ab1df02d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:09.922305 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a96eca-b396-4ea9-a869-585eccb83433 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:09.922674 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a96eca-b396-4ea9-a869-585eccb83433 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:09.923022 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a96eca-b396-4ea9-a869-585eccb83433 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:09.924672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 266/540] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:46:09.930043 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81fd889-4499-44b8-8a28-9cc99d73bd34 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:09.930334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81fd889-4499-44b8-8a28-9cc99d73bd34 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:09.930612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81fd889-4499-44b8-8a28-9cc99d73bd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:09.959438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3ebd6-6867-4934-93f1-a66766b3b0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:09.960150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3ebd6-6867-4934-93f1-a66766b3b0c6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:09.960614 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3ebd6-6867-4934-93f1-a66766b3b0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:09.960969 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3ebd6-6867-4934-93f1-a66766b3b0c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:09.961848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3ebd6-6867-4934-93f1-a66766b3b0c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:09.962256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3ebd6-6867-4934-93f1-a66766b3b0c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:09.963798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3ebd6-6867-4934-93f1-a66766b3b0c6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:09.976131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 275/541] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:09.978971 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a81fd889-4499-44b8-8a28-9cc99d73bd34 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:09.980280 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a81fd889-4499-44b8-8a28-9cc99d73bd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:09.983147 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3ebd6-6867-4934-93f1-a66766b3b0c6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:10.116196 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3ebd6-6867-4934-93f1-a66766b3b0c6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:10.154807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 267/542] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:46:10.157037 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db54a30-7bd7-40da-a5cd-b965202cb095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:10.157817 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db54a30-7bd7-40da-a5cd-b965202cb095 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:10.158256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db54a30-7bd7-40da-a5cd-b965202cb095 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:10.158677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db54a30-7bd7-40da-a5cd-b965202cb095 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:10.159583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db54a30-7bd7-40da-a5cd-b965202cb095 None admin] RBAC: Adding query filter params (role.id=d1bb55f25dff4694b8675e0ef5fc4ec7, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=13f1840d92d94c3ebd16000a4889c377) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:10.160041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db54a30-7bd7-40da-a5cd-b965202cb095 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:10.161635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db54a30-7bd7-40da-a5cd-b965202cb095 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:10.171122 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 276/543] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:46:09 2022] GET /identity/v3/role_assignments?user.id=13f1840d92d94c3ebd16000a4889c377&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:10.488183 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81fd889-4499-44b8-8a28-9cc99d73bd34 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:10.534600 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:10.615808 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81fd889-4499-44b8-8a28-9cc99d73bd34 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:10.671960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8f91b-8b95-4fd3-9489-8353f9bb8fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:10.672606 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8f91b-8b95-4fd3-9489-8353f9bb8fcf None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:10.672884 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8f91b-8b95-4fd3-9489-8353f9bb8fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:10.673154 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8f91b-8b95-4fd3-9489-8353f9bb8fcf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:10.685734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8f91b-8b95-4fd3-9489-8353f9bb8fcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:10.690562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-13a8f91b-8b95-4fd3-9489-8353f9bb8fcf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:10.690562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:10.690562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:10.690562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:10.690562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:10.690562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:10.690562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:10.690562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:10.690562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:10.690562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:10.690562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:10.690562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:10.690562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:10.690562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:10.690562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:10.690562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:10.691418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:10.691418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:10.691418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:10.691418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:10.691418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:10.691418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:10.691418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:10.691418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:10.691418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:10.691418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:10.691418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:10.691418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:10.691418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:10.691418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:10.691418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:10.691418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:10.692198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:10.692198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:10.692198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:10.692198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:10.692198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:10.692198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:10.692198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:10.692198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:10.692198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:10.692198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:10.692198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:10.692198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:10.694300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 268/544] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:10.699319 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db54a30-7bd7-40da-a5cd-b965202cb095 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:10.726521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:10.753057 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92b393-51f9-43fb-ae4a-64511cbe2254 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:10.753325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92b393-51f9-43fb-ae4a-64511cbe2254 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:10.753531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92b393-51f9-43fb-ae4a-64511cbe2254 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:10.754925 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 269/545] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:46:10.760322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1250ae3-27a3-4417-b546-ef589aede064 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:10.760612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1250ae3-27a3-4417-b546-ef589aede064 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:10.760889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1250ae3-27a3-4417-b546-ef589aede064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:10.780037 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-b1250ae3-27a3-4417-b546-ef589aede064 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:10.781221 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b1250ae3-27a3-4417-b546-ef589aede064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:10.858399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db54a30-7bd7-40da-a5cd-b965202cb095 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:10.924201 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6e935-12e3-4110-917b-b989096a18dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:10.925390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 270/546] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:46:10.926025 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6e935-12e3-4110-917b-b989096a18dd None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:10.933946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6e935-12e3-4110-917b-b989096a18dd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:10.934558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6e935-12e3-4110-917b-b989096a18dd None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:10.935454 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af6e935-12e3-4110-917b-b989096a18dd None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:10.935915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af6e935-12e3-4110-917b-b989096a18dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:10.937546 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af6e935-12e3-4110-917b-b989096a18dd None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:10.948141 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 277/547] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:46:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:10.954008 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1250ae3-27a3-4417-b546-ef589aede064 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:11.004938 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:11.104621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1250ae3-27a3-4417-b546-ef589aede064 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:11.148276 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf96ada-c504-4e27-8b9f-1d8635315f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:11.148914 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf96ada-c504-4e27-8b9f-1d8635315f9c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:11.149196 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf96ada-c504-4e27-8b9f-1d8635315f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:11.149474 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf96ada-c504-4e27-8b9f-1d8635315f9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:11.162730 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf96ada-c504-4e27-8b9f-1d8635315f9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:11.164174 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf96ada-c504-4e27-8b9f-1d8635315f9c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:11.188071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-ecf96ada-c504-4e27-8b9f-1d8635315f9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:11.188071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:11.188071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:11.188071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:11.188071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:11.188071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:11.188071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:11.188071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:11.188071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:11.188071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:11.188071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:11.188071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:11.188071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:11.188071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:11.188071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:11.188071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:11.189254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:11.189254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:11.189254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:11.189254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:11.189254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:11.189254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:11.189254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:11.189254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:11.189254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:11.189254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:11.189254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:11.189254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:11.189254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:11.189254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:11.189254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:11.189254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:11.189254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:11.190569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:11.190569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:11.190569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:11.190569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 271/548] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:11.194952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6e935-12e3-4110-917b-b989096a18dd None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:11.336022 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf96ada-c504-4e27-8b9f-1d8635315f9c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:11.338328 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:11.374085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6e935-12e3-4110-917b-b989096a18dd None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:11.432302 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:11.438811 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3590e48-1005-4f2f-b1e0-6b1fd874afe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:11.439425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3590e48-1005-4f2f-b1e0-6b1fd874afe8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:11.439703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3590e48-1005-4f2f-b1e0-6b1fd874afe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:11.439974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3590e48-1005-4f2f-b1e0-6b1fd874afe8 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:11.440616 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3590e48-1005-4f2f-b1e0-6b1fd874afe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:11.440914 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3590e48-1005-4f2f-b1e0-6b1fd874afe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:11.442332 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3590e48-1005-4f2f-b1e0-6b1fd874afe8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:11.454769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf96ada-c504-4e27-8b9f-1d8635315f9c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:11.462831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 278/549] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:11.468657 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3590e48-1005-4f2f-b1e0-6b1fd874afe8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:11.495449 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0717a4-1ebc-4b0a-bb59-c9af89e5fd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:11.496660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0717a4-1ebc-4b0a-bb59-c9af89e5fd8c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:11.497163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0717a4-1ebc-4b0a-bb59-c9af89e5fd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:11.497667 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0717a4-1ebc-4b0a-bb59-c9af89e5fd8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:11.508818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0717a4-1ebc-4b0a-bb59-c9af89e5fd8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:11.514905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-6e0717a4-1ebc-4b0a-bb59-c9af89e5fd8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:11.514905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:11.514905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:11.514905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:11.514905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:11.514905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:11.514905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:11.514905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:11.514905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:11.514905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:11.514905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:11.514905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:11.514905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:11.514905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:11.514905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:11.514905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:11.515830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:11.515830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:11.515830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:11.515830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:11.515830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:11.515830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:11.515830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:11.515830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:11.515830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:11.515830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:11.515830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:11.515830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:11.515830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:11.515830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:11.515830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:11.515830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:11.516657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:11.516657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:11.516657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:11.516657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:11.516657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:11.516657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:11.516657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:11.516657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:11.516657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:11.516657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:11.516657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:11.516657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:11.520410 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 272/550] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:11.526170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0717a4-1ebc-4b0a-bb59-c9af89e5fd8c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:11.652275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0717a4-1ebc-4b0a-bb59-c9af89e5fd8c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:11.692851 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc027dae-76ac-4506-8297-4ef27b10e2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:11.693459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc027dae-76ac-4506-8297-4ef27b10e2fb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:11.693736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc027dae-76ac-4506-8297-4ef27b10e2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:11.694166 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc027dae-76ac-4506-8297-4ef27b10e2fb None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:11.695137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc027dae-76ac-4506-8297-4ef27b10e2fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:11.695246 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc027dae-76ac-4506-8297-4ef27b10e2fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:11.696561 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc027dae-76ac-4506-8297-4ef27b10e2fb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:11.697965 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3590e48-1005-4f2f-b1e0-6b1fd874afe8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:11.705278 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 273/551] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:46:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:11.706709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473136c-232d-414b-94c6-eb6ab9da4e3e None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:11.706998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473136c-232d-414b-94c6-eb6ab9da4e3e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:11.707275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473136c-232d-414b-94c6-eb6ab9da4e3e None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:11.708573 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 274/552] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:11.713427 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473136c-232d-414b-94c6-eb6ab9da4e3e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:11.716466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:11.738662 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294be678-a442-4202-9e04-ca38fadba961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:11.739744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294be678-a442-4202-9e04-ca38fadba961 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:11.740584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294be678-a442-4202-9e04-ca38fadba961 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:11.741307 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294be678-a442-4202-9e04-ca38fadba961 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:11.752190 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294be678-a442-4202-9e04-ca38fadba961 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:11.753709 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294be678-a442-4202-9e04-ca38fadba961 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:11.769164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-294be678-a442-4202-9e04-ca38fadba961 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 11:46:11.769164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:11.769164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:11.769164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:11.769164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:11.769164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:11.769164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:11.769164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:11.769164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:11.769164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:11.769164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:11.769164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:11.769164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:11.769164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:11.769164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:11.769164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:11.770335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:11.770335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:11.770335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:11.770335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:11.770335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:11.770335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:11.770335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:11.770335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:11.770335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:11.770335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:11.770335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:11.770335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:11.770335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:11.770335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:11.770335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:11.770335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:11.770335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:11.770335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:11.771471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:11.771471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:11.771471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:11.771471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:11.771471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:11.771471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:11.771471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:11.771471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:11.771471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 11:46:11.771471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:11.773846 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 279/553] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:46:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:11.775947 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c65c9-6b1b-4e49-a268-ea24e17cae6f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:11.776432 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c65c9-6b1b-4e49-a268-ea24e17cae6f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:11.776848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c65c9-6b1b-4e49-a268-ea24e17cae6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:11.784595 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:11.801241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ea7c65c9-6b1b-4e49-a268-ea24e17cae6f None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:11.802857 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ea7c65c9-6b1b-4e49-a268-ea24e17cae6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:11.843142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473136c-232d-414b-94c6-eb6ab9da4e3e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:11.886306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6849d7b-639d-479c-bff1-98c728f7a635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:11.887565 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6849d7b-639d-479c-bff1-98c728f7a635 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:11.888011 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6849d7b-639d-479c-bff1-98c728f7a635 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:11.888392 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6849d7b-639d-479c-bff1-98c728f7a635 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:11.899097 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 280/554] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:11.905710 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c65c9-6b1b-4e49-a268-ea24e17cae6f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:11.906395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6849d7b-639d-479c-bff1-98c728f7a635 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:11.908064 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6849d7b-639d-479c-bff1-98c728f7a635 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:11.940144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d6849d7b-639d-479c-bff1-98c728f7a635 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:11.940144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:11.940144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:11.940144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:11.940144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:11.940144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:11.940144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:11.940144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:11.940144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:11.940144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:11.940144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:11.940144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:11.940144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:11.940144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:11.940144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:11.940144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:11.956111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:11.956111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:11.956111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:11.956111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:11.956111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:11.956111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:11.956111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:11.956111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:11.956111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:11.956111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:11.956111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:11.956111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:11.956111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:11.956111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:11.956111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:11.956111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:11.956111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:11.956965 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:11.956965 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:11.956965 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:11.956965 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 275/555] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:11.956965 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6849d7b-639d-479c-bff1-98c728f7a635 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:12.019464 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:12.111935 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6849d7b-639d-479c-bff1-98c728f7a635 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:12.145447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c65c9-6b1b-4e49-a268-ea24e17cae6f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:12.183225 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b9a80-21f4-44b0-8d41-fd2ccad4c4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:12.183890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b9a80-21f4-44b0-8d41-fd2ccad4c4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:12.184074 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b9a80-21f4-44b0-8d41-fd2ccad4c4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:12.184316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b9a80-21f4-44b0-8d41-fd2ccad4c4a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:12.185012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b9a80-21f4-44b0-8d41-fd2ccad4c4a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:12.185317 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b9a80-21f4-44b0-8d41-fd2ccad4c4a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:12.186747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b9a80-21f4-44b0-8d41-fd2ccad4c4a2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:12.211290 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:12.216846 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 276/556] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:12.217723 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ab9ee-aa3c-4728-9eaf-00c25d9341f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:12.218195 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ab9ee-aa3c-4728-9eaf-00c25d9341f4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:12.218479 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ab9ee-aa3c-4728-9eaf-00c25d9341f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:12.218756 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ab9ee-aa3c-4728-9eaf-00c25d9341f4 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:12.219501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203ab9ee-aa3c-4728-9eaf-00c25d9341f4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:12.219811 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203ab9ee-aa3c-4728-9eaf-00c25d9341f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:12.221400 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203ab9ee-aa3c-4728-9eaf-00c25d9341f4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:12.221846 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b9a80-21f4-44b0-8d41-fd2ccad4c4a2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:12.241263 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 281/557] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Sep 14 11:46:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:12.242151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ab9ee-aa3c-4728-9eaf-00c25d9341f4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:12.336673 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b9a80-21f4-44b0-8d41-fd2ccad4c4a2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:12.349300 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ab9ee-aa3c-4728-9eaf-00c25d9341f4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:12.353582 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:12.375245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a227d34-a4d4-4bbd-89c2-f2428bf15965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:12.376192 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a227d34-a4d4-4bbd-89c2-f2428bf15965 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:12.376752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a227d34-a4d4-4bbd-89c2-f2428bf15965 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:12.377032 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a227d34-a4d4-4bbd-89c2-f2428bf15965 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:12.383262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d79b0-4487-41a0-a425-15a5a500a7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:12.383895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d79b0-4487-41a0-a425-15a5a500a7fe None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:12.384180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a227d34-a4d4-4bbd-89c2-f2428bf15965 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:12.384280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d79b0-4487-41a0-a425-15a5a500a7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:12.384748 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d79b0-4487-41a0-a425-15a5a500a7fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:12.385621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a227d34-a4d4-4bbd-89c2-f2428bf15965 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:12.395413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d79b0-4487-41a0-a425-15a5a500a7fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:12.399625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-6a227d34-a4d4-4bbd-89c2-f2428bf15965 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 11:46:12.399625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:12.399625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:12.399625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:12.399625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:12.399625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:12.399625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:12.399625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:12.399625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:12.399625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:12.399625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:12.399625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:12.399625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:12.399625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:12.399625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:12.399625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:12.400563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:12.400563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:12.400563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:12.400563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:12.400563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:12.400563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:12.400563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:12.400563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:12.400563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:12.400563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:12.400563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:12.400563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:12.400563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:12.400563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:12.400563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:12.400563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:12.400563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:12.400563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:12.401503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:12.401503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:12.401503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:12.401503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:12.401503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:12.401503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:12.401503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:12.401503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:12.401503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 11:46:12.401503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:12.402103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-754d79b0-4487-41a0-a425-15a5a500a7fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:12.402103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:12.402103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:12.402103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:12.402103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:12.402103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:12.402103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:12.402103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:12.402103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:12.402103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:12.402103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:12.402103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:12.402103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:12.402103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:12.402103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:12.402103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:12.402948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:12.402948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:12.402948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:12.402948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:12.402948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:12.402948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:12.402948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:12.402948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:12.402948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:12.402948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:12.402948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:12.402948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:12.402948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:12.402948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:12.402948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:12.402948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:12.404411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:12.404411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:12.404411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:12.404411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:12.404411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:12.404411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:12.404411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:12.404411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:12.404411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:12.404411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:12.404411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:12.404411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:12.407504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 277/558] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Sep 14 11:46:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:12.407504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a227d34-a4d4-4bbd-89c2-f2428bf15965 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:12.407796 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 282/559] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:12.407796 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95b902-5b07-46e7-adc7-10c3fee7bf1f None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:12.407796 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95b902-5b07-46e7-adc7-10c3fee7bf1f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:12.408216 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95b902-5b07-46e7-adc7-10c3fee7bf1f None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:12.413381 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 283/560] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:12.414150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95b902-5b07-46e7-adc7-10c3fee7bf1f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:12.467371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:12.512538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a227d34-a4d4-4bbd-89c2-f2428bf15965 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:12.522400 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95b902-5b07-46e7-adc7-10c3fee7bf1f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:12.551907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378228b6-1f47-4938-b035-55d0d7eeed66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:12.552513 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378228b6-1f47-4938-b035-55d0d7eeed66 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:12.552791 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378228b6-1f47-4938-b035-55d0d7eeed66 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:12.553064 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378228b6-1f47-4938-b035-55d0d7eeed66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:12.559256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172bb95a-cd07-418f-ab83-d0d65922cf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:12.559889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bb95a-cd07-418f-ab83-d0d65922cf8a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:12.560173 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bb95a-cd07-418f-ab83-d0d65922cf8a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:12.560451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bb95a-cd07-418f-ab83-d0d65922cf8a None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:12.561178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172bb95a-cd07-418f-ab83-d0d65922cf8a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:12.561487 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172bb95a-cd07-418f-ab83-d0d65922cf8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:12.562971 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172bb95a-cd07-418f-ab83-d0d65922cf8a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:12.582198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 284/561] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Sep 14 11:46:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:12.583343 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172bb95a-cd07-418f-ab83-d0d65922cf8a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:12.609596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:12.635931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378228b6-1f47-4938-b035-55d0d7eeed66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:12.639999 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378228b6-1f47-4938-b035-55d0d7eeed66 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:12.701509 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172bb95a-cd07-418f-ab83-d0d65922cf8a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:12.711499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:12.736321 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778d79c-8078-4c1d-a228-8f80ab2bd196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:12.736849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778d79c-8078-4c1d-a228-8f80ab2bd196 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:12.737136 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778d79c-8078-4c1d-a228-8f80ab2bd196 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:12.737407 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778d79c-8078-4c1d-a228-8f80ab2bd196 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:12.738092 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778d79c-8078-4c1d-a228-8f80ab2bd196 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:12.738412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778d79c-8078-4c1d-a228-8f80ab2bd196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:12.739770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778d79c-8078-4c1d-a228-8f80ab2bd196 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:12.745577 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 278/562] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:46:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:46:12.746850 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba1d2d-897e-4399-9fdb-66aadf6111ab None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:12.747138 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba1d2d-897e-4399-9fdb-66aadf6111ab None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:12.747941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba1d2d-897e-4399-9fdb-66aadf6111ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:12.752777 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 285/563] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:46:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:12.752777 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778d79c-8078-4c1d-a228-8f80ab2bd196 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:12.768118 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d5ba1d2d-897e-4399-9fdb-66aadf6111ab None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:12.769308 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d5ba1d2d-897e-4399-9fdb-66aadf6111ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:12.838830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 279/564] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:12.840281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be379255-ea60-4a85-8b6d-c7a6eeb8b421 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:12.840661 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be379255-ea60-4a85-8b6d-c7a6eeb8b421 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:12.841048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be379255-ea60-4a85-8b6d-c7a6eeb8b421 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:12.842514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 280/565] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:12.847085 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be379255-ea60-4a85-8b6d-c7a6eeb8b421 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:12.851195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:12.852986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778d79c-8078-4c1d-a228-8f80ab2bd196 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:12.890812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ad39f-2a56-44bf-9681-98285a94e1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:12.891167 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ad39f-2a56-44bf-9681-98285a94e1fc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:12.891512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ad39f-2a56-44bf-9681-98285a94e1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:12.891848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ad39f-2a56-44bf-9681-98285a94e1fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:12.902399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ad39f-2a56-44bf-9681-98285a94e1fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:12.904871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ad39f-2a56-44bf-9681-98285a94e1fc None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:12.920365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-104ad39f-2a56-44bf-9681-98285a94e1fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:12.920365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:12.920365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:12.920365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:12.920365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:12.920365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:12.920365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:12.920365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:12.920365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:12.920365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:12.920365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:12.920365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:12.920365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:12.920365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:12.920365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:12.920365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:12.921501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:12.921501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:12.921501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:12.921501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:12.921501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:12.921501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:12.921501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:12.921501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:12.921501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:12.921501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:12.921501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:12.921501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:12.921501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:12.921501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:12.921501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:12.921501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:12.921501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:12.922640 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:12.922640 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:12.922640 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:12.922856 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 286/566] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:12.923672 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ad39f-2a56-44bf-9681-98285a94e1fc None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:12.948121 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:12.961519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be379255-ea60-4a85-8b6d-c7a6eeb8b421 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:12.999552 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6799c7-ea54-452c-b767-b691f11b6494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:13.000243 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6799c7-ea54-452c-b767-b691f11b6494 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:13.001045 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6799c7-ea54-452c-b767-b691f11b6494 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:13.001328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6799c7-ea54-452c-b767-b691f11b6494 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:13.014491 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6799c7-ea54-452c-b767-b691f11b6494 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:13.016003 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6799c7-ea54-452c-b767-b691f11b6494 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:13.025497 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ad39f-2a56-44bf-9681-98285a94e1fc None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:13.031415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-ca6799c7-ea54-452c-b767-b691f11b6494 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:13.031415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:13.031415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:13.031415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:13.031415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:13.031415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:13.031415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:13.031415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:13.031415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:13.031415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:13.031415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:13.031415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:13.031415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:13.031415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:13.031415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:13.031415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:13.032386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:13.032386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:13.032386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:13.032386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:13.032386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:13.032386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:13.032386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:13.032386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:13.032386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:13.032386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:13.032386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:13.032386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:13.032386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:13.032386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:13.032386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:13.032386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:13.032386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:13.033270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:13.033270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:13.033270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:13.033270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 281/567] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:13.033919 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b816e80-2ad4-4ce0-8366-717840fbf6bc None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:13.034084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b816e80-2ad4-4ce0-8366-717840fbf6bc None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:13.034332 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b816e80-2ad4-4ce0-8366-717840fbf6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:13.052690 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2b816e80-2ad4-4ce0-8366-717840fbf6bc None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:13.054036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2b816e80-2ad4-4ce0-8366-717840fbf6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:13.064068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29c1ff-83a0-45f7-baaa-0ef0ed6be66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:13.064694 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29c1ff-83a0-45f7-baaa-0ef0ed6be66a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:13.064981 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29c1ff-83a0-45f7-baaa-0ef0ed6be66a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:13.065257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29c1ff-83a0-45f7-baaa-0ef0ed6be66a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:13.076677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29c1ff-83a0-45f7-baaa-0ef0ed6be66a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:13.078154 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29c1ff-83a0-45f7-baaa-0ef0ed6be66a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:13.091936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2b29c1ff-83a0-45f7-baaa-0ef0ed6be66a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:13.091936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:13.091936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:13.091936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:13.091936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:13.091936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:13.091936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:13.091936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:13.091936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:13.091936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:13.091936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:13.091936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:13.091936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:13.091936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:13.091936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:13.091936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:13.092873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:13.092873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:13.092873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:13.092873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:13.092873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:13.092873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:13.092873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:13.092873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:13.092873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:13.092873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:13.092873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:13.092873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:13.092873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:13.092873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:13.092873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:13.092873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:13.092873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:13.093784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:13.093784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:13.093784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:13.093784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 287/568] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:13.093784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29c1ff-83a0-45f7-baaa-0ef0ed6be66a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:13.114472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:13.131915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 282/569] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:13.132730 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b816e80-2ad4-4ce0-8366-717840fbf6bc None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:13.175007 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:13.200709 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29c1ff-83a0-45f7-baaa-0ef0ed6be66a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:13.237154 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b775f60-2eae-4889-9fee-6145966ecc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:13.237759 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b775f60-2eae-4889-9fee-6145966ecc65 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:13.238077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b775f60-2eae-4889-9fee-6145966ecc65 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:13.238357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b775f60-2eae-4889-9fee-6145966ecc65 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:13.239217 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b775f60-2eae-4889-9fee-6145966ecc65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:13.239552 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b775f60-2eae-4889-9fee-6145966ecc65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:13.240954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b775f60-2eae-4889-9fee-6145966ecc65 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:13.241288 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b816e80-2ad4-4ce0-8366-717840fbf6bc None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:13.253359 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 288/570] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:13.254128 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b775f60-2eae-4889-9fee-6145966ecc65 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:13.284626 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a94124-1ace-403a-9ba0-3ce1502ee22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:13.285439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a94124-1ace-403a-9ba0-3ce1502ee22f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:13.285916 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a94124-1ace-403a-9ba0-3ce1502ee22f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:13.286331 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a94124-1ace-403a-9ba0-3ce1502ee22f None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:13.287107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a94124-1ace-403a-9ba0-3ce1502ee22f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:13.287552 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a94124-1ace-403a-9ba0-3ce1502ee22f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:13.289939 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a94124-1ace-403a-9ba0-3ce1502ee22f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:13.304119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 283/571] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:13.304927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a94124-1ace-403a-9ba0-3ce1502ee22f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:13.355652 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:13.361533 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b775f60-2eae-4889-9fee-6145966ecc65 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:13.401423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e52220-62cb-44d9-a51c-4a7d2eef3a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:13.402060 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e52220-62cb-44d9-a51c-4a7d2eef3a3c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:13.402337 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e52220-62cb-44d9-a51c-4a7d2eef3a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:13.402607 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e52220-62cb-44d9-a51c-4a7d2eef3a3c None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:13.403247 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e52220-62cb-44d9-a51c-4a7d2eef3a3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:13.403584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e52220-62cb-44d9-a51c-4a7d2eef3a3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:13.404945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e52220-62cb-44d9-a51c-4a7d2eef3a3c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:13.406469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:13.413300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a94124-1ace-403a-9ba0-3ce1502ee22f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:13.416736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 289/572] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:13.417528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e52220-62cb-44d9-a51c-4a7d2eef3a3c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:13.447955 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f4661-74b6-4f5c-9851-e52cd3d660f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:13.448578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f4661-74b6-4f5c-9851-e52cd3d660f2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:13.448862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f4661-74b6-4f5c-9851-e52cd3d660f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:13.449136 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f4661-74b6-4f5c-9851-e52cd3d660f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:13.458974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f4661-74b6-4f5c-9851-e52cd3d660f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:13.464176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-499f4661-74b6-4f5c-9851-e52cd3d660f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:13.464176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:13.464176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:13.464176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:13.464176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:13.464176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:13.464176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:13.464176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:13.464176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:13.464176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:13.464176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:13.464176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:13.464176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:13.464176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:13.464176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:13.464176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:13.465094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:13.465094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:13.465094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:13.465094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:13.465094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:13.465094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:13.465094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:13.465094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:13.465094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:13.465094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:13.465094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:13.465094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:13.465094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:13.465094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:13.465094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:13.465094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:13.466578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:13.466578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:13.466578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:13.466578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:13.466578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:13.466578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:13.466578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:13.466578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:13.466578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:13.466578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:13.466578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:13.466578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:13.466578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 284/573] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:13.466578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f4661-74b6-4f5c-9851-e52cd3d660f2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:13.518231 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e52220-62cb-44d9-a51c-4a7d2eef3a3c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:13.551686 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8e8fa-ada6-4339-a9d0-fe5b697bb99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:13.552303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8e8fa-ada6-4339-a9d0-fe5b697bb99c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:13.552583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8e8fa-ada6-4339-a9d0-fe5b697bb99c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:13.552861 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8e8fa-ada6-4339-a9d0-fe5b697bb99c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:13.559273 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b8e8fa-ada6-4339-a9d0-fe5b697bb99c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:13.560730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b8e8fa-ada6-4339-a9d0-fe5b697bb99c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:13.564141 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f4661-74b6-4f5c-9851-e52cd3d660f2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:13.570863 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:13.571575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-b8b8e8fa-ada6-4339-a9d0-fe5b697bb99c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:13.571575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:13.571575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:13.571575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:13.571575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:13.571575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:13.571575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:13.571575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:13.571575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:13.571575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:13.571575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:13.571575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:13.571575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:13.571575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:13.571575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:13.572584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:13.572584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:13.572584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:13.572584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:13.572584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:13.572584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:13.572584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:13.572584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:13.572584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:13.572584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:13.572584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:13.572584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:13.572584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:13.572584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:13.572584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:13.572584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:13.572584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:13.580808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:13.580808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:13.580808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:13.580808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:13.580808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:13.580808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:13.580808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:13.580808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:13.580808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 290/574] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:13.580808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8e8fa-ada6-4339-a9d0-fe5b697bb99c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:13.597685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f60ddb-14a9-4305-b4d1-21d9888f913d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:13.598344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f60ddb-14a9-4305-b4d1-21d9888f913d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:13.598626 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f60ddb-14a9-4305-b4d1-21d9888f913d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:13.598895 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f60ddb-14a9-4305-b4d1-21d9888f913d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:13.605736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f60ddb-14a9-4305-b4d1-21d9888f913d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:13.608157 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f60ddb-14a9-4305-b4d1-21d9888f913d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:13.619499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-88f60ddb-14a9-4305-b4d1-21d9888f913d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:13.619499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:13.619499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:13.619499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:13.619499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:13.619499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:13.619499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:13.619499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:13.619499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:13.619499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:13.619499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:13.619499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:13.619499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:13.619499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:13.619499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:13.620550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:13.620550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:13.620550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:13.620550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:13.620550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:13.620550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:13.620550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:13.620550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:13.620550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:13.620550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:13.620550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:13.620550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:13.620550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:13.620550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:13.620550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:13.620550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:13.620550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:13.621487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:13.621487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:13.621487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:13.621487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:13.621487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:13.621487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:13.621487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:13.621487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:13.621487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 285/575] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:13.622053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f60ddb-14a9-4305-b4d1-21d9888f913d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:13.634605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:13.680546 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8e8fa-ada6-4339-a9d0-fe5b697bb99c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:13.712239 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b4de7-e3e4-4a41-bfde-ca058aab3c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:13.712854 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b4de7-e3e4-4a41-bfde-ca058aab3c9d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:13.713132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b4de7-e3e4-4a41-bfde-ca058aab3c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:13.713411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b4de7-e3e4-4a41-bfde-ca058aab3c9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:13.719468 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f60ddb-14a9-4305-b4d1-21d9888f913d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:13.723641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7b4de7-e3e4-4a41-bfde-ca058aab3c9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:13.728742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-6e7b4de7-e3e4-4a41-bfde-ca058aab3c9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:13.728742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:13.728742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:13.728742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:13.728742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:13.728742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:13.728742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:13.728742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:13.728742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:13.728742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:13.728742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:13.728742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:13.728742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:13.728742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:13.728742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:13.728742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:13.729838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:13.729838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:13.729838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:13.729838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:13.729838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:13.729838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:13.729838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:13.729838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:13.729838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:13.729838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:13.729838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:13.729838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:13.729838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:13.729838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:13.729838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:13.729838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:13.730901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:13.730901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:13.730901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:13.730901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:13.730901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:13.730901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:13.730901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:13.730901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:13.730901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:13.730901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:13.730901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:13.730901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:13.735347 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 291/576] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:13.736008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b4de7-e3e4-4a41-bfde-ca058aab3c9d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:13.754043 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fadca7-10de-411f-b215-1595ac6e3332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:13.754670 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fadca7-10de-411f-b215-1595ac6e3332 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:13.754950 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fadca7-10de-411f-b215-1595ac6e3332 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:13.755223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fadca7-10de-411f-b215-1595ac6e3332 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:13.762482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fadca7-10de-411f-b215-1595ac6e3332 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:13.763929 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fadca7-10de-411f-b215-1595ac6e3332 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:13.774998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-b6fadca7-10de-411f-b215-1595ac6e3332 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 14 11:46:13.774998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:13.774998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:13.774998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:13.774998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:13.774998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:13.774998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:13.774998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:13.774998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:13.774998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:13.774998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:13.774998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:13.774998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:13.774998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:13.774998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:13.774998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:13.781170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:13.781170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:13.781170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:13.781170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:13.781170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:13.781170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:13.781170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:13.781170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:13.781170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:13.781170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:13.781170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:13.781170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:13.781170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:13.781170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:13.781170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:13.781170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:13.781170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:13.781170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:13.782437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:13.782437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:13.782437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:13.782437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:13.782437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:13.782437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:13.782437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:13.782437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:13.782437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 14 11:46:13.782437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:13.782437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 286/577] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:46:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:13.782437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fadca7-10de-411f-b215-1595ac6e3332 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:13.804874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:13.835130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b4de7-e3e4-4a41-bfde-ca058aab3c9d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:13.849269 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:13.868927 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcb863-ebc0-4c9e-91c6-841e3d42a6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:13.869532 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcb863-ebc0-4c9e-91c6-841e3d42a6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:13.869816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcb863-ebc0-4c9e-91c6-841e3d42a6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:13.870126 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcb863-ebc0-4c9e-91c6-841e3d42a6e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:13.870795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bcb863-ebc0-4c9e-91c6-841e3d42a6e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:13.871105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bcb863-ebc0-4c9e-91c6-841e3d42a6e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:13.872542 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bcb863-ebc0-4c9e-91c6-841e3d42a6e9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:13.878922 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fadca7-10de-411f-b215-1595ac6e3332 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:13.880841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 292/578] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:46:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:13.881627 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcb863-ebc0-4c9e-91c6-841e3d42a6e9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:13.915571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26941c-60a3-4a03-a116-d102cf3f8703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:13.916349 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26941c-60a3-4a03-a116-d102cf3f8703 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:13.916739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26941c-60a3-4a03-a116-d102cf3f8703 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:13.917115 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26941c-60a3-4a03-a116-d102cf3f8703 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:13.917945 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26941c-60a3-4a03-a116-d102cf3f8703 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:13.918354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26941c-60a3-4a03-a116-d102cf3f8703 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:13.919888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26941c-60a3-4a03-a116-d102cf3f8703 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:13.931645 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 287/579] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:13.932569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26941c-60a3-4a03-a116-d102cf3f8703 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:14.006773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcb863-ebc0-4c9e-91c6-841e3d42a6e9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:14.014125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:14.028079 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26941c-60a3-4a03-a116-d102cf3f8703 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:14.039712 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388abd8-f845-4def-a4f9-6b766a5cc78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:14.043641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388abd8-f845-4def-a4f9-6b766a5cc78f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:14.043641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388abd8-f845-4def-a4f9-6b766a5cc78f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:14.043641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388abd8-f845-4def-a4f9-6b766a5cc78f None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:14.043641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388abd8-f845-4def-a4f9-6b766a5cc78f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:14.043641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388abd8-f845-4def-a4f9-6b766a5cc78f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:14.043641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388abd8-f845-4def-a4f9-6b766a5cc78f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:14.055064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 293/580] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:14.055849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388abd8-f845-4def-a4f9-6b766a5cc78f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:14.062824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec1370-5678-4ef1-8814-0f47841edaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:14.063437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec1370-5678-4ef1-8814-0f47841edaff None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:14.064136 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec1370-5678-4ef1-8814-0f47841edaff None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:14.064417 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec1370-5678-4ef1-8814-0f47841edaff None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:14.065084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec1370-5678-4ef1-8814-0f47841edaff None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:14.065391 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec1370-5678-4ef1-8814-0f47841edaff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:14.067147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec1370-5678-4ef1-8814-0f47841edaff None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:14.075031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 288/581] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:46:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:14.076330 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec1370-5678-4ef1-8814-0f47841edaff None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:14.093658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:14.159143 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388abd8-f845-4def-a4f9-6b766a5cc78f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:14.174847 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec1370-5678-4ef1-8814-0f47841edaff None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:14.191618 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c397aab-63d8-44d4-a743-7a47634cc1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:14.192208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c397aab-63d8-44d4-a743-7a47634cc1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:14.192488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c397aab-63d8-44d4-a743-7a47634cc1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:14.192762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c397aab-63d8-44d4-a743-7a47634cc1a3 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:14.193486 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c397aab-63d8-44d4-a743-7a47634cc1a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:14.193792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c397aab-63d8-44d4-a743-7a47634cc1a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:14.195251 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c397aab-63d8-44d4-a743-7a47634cc1a3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:14.209358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35973e3f-f3c0-4149-afbf-8e5607bf17ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:14.210019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35973e3f-f3c0-4149-afbf-8e5607bf17ac None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:14.210450 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35973e3f-f3c0-4149-afbf-8e5607bf17ac None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:14.210727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35973e3f-f3c0-4149-afbf-8e5607bf17ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:14.212150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 294/582] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Sep 14 11:46:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:14.213356 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c397aab-63d8-44d4-a743-7a47634cc1a3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:14.221606 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35973e3f-f3c0-4149-afbf-8e5607bf17ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:14.223087 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35973e3f-f3c0-4149-afbf-8e5607bf17ac None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:14.230942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:14.237226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-35973e3f-f3c0-4149-afbf-8e5607bf17ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:14.237226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:14.237226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:14.237226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:14.237226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:14.237226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:14.237226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:14.237226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:14.237226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:14.237226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:14.237226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:14.237226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:14.237226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:14.237226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:14.237226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:14.237226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:14.238266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:14.238266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:14.238266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:14.238266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:14.238266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:14.238266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:14.238266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:14.238266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:14.238266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:14.238266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:14.238266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:14.238266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:14.238266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:14.238266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:14.238266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:14.238266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:14.238266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:14.239165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:14.239165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:14.239165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:14.239165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 289/583] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:14.239165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35973e3f-f3c0-4149-afbf-8e5607bf17ac None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:14.307312 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:14.314780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c397aab-63d8-44d4-a743-7a47634cc1a3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:14.338371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35973e3f-f3c0-4149-afbf-8e5607bf17ac None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:14.348157 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba0fb1-6534-4603-9fb8-799cddb0d5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:14.348811 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba0fb1-6534-4603-9fb8-799cddb0d5d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:14.349090 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba0fb1-6534-4603-9fb8-799cddb0d5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:14.349364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba0fb1-6534-4603-9fb8-799cddb0d5d9 None admin] PATH_INFO: `/v3/projects/449a5fac1acd478abea544446c08ecd9/users/38b093bc4c1d4ebcbd2a6bb16b70c029/roles/d1bb55f25dff4694b8675e0ef5fc4ec7` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:14.363134 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba0fb1-6534-4603-9fb8-799cddb0d5d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=449a5fac1acd478abea544446c08ecd9, user_id=38b093bc4c1d4ebcbd2a6bb16b70c029, role_id=d1bb55f25dff4694b8675e0ef5fc4ec7)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:14.364560 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba0fb1-6534-4603-9fb8-799cddb0d5d9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:14.371112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65a094-b852-47b2-9f90-ddddc34283d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:14.371706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65a094-b852-47b2-9f90-ddddc34283d6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:14.371983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65a094-b852-47b2-9f90-ddddc34283d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:14.372783 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65a094-b852-47b2-9f90-ddddc34283d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:14.373495 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65a094-b852-47b2-9f90-ddddc34283d6 None admin] RBAC: Adding query filter params (role.id=d1bb55f25dff4694b8675e0ef5fc4ec7, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=d1047c7d18c1466d989cd895117cd807) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:14.373799 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65a094-b852-47b2-9f90-ddddc34283d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:14.375223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65a094-b852-47b2-9f90-ddddc34283d6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:14.380421 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 290/584] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:46:14 2022] GET /identity/v3/role_assignments?user.id=d1047c7d18c1466d989cd895117cd807&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:14.381260 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65a094-b852-47b2-9f90-ddddc34283d6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:14.437902 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 295/585] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:46:14 2022] PUT /identity/v3/projects/449a5fac1acd478abea544446c08ecd9/users/38b093bc4c1d4ebcbd2a6bb16b70c029/roles/d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:46:14.437902 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba0fb1-6534-4603-9fb8-799cddb0d5d9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:14.468174 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:14.518409 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65a094-b852-47b2-9f90-ddddc34283d6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:14.567035 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b419601-badb-4dd3-9416-4461d3fc678c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:14.567923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b419601-badb-4dd3-9416-4461d3fc678c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:14.569037 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b419601-badb-4dd3-9416-4461d3fc678c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:14.569318 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b419601-badb-4dd3-9416-4461d3fc678c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:14.584744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b419601-badb-4dd3-9416-4461d3fc678c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:14.586256 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b419601-badb-4dd3-9416-4461d3fc678c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:14.601750 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba0fb1-6534-4603-9fb8-799cddb0d5d9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:14.607211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-8b419601-badb-4dd3-9416-4461d3fc678c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:14.607211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:14.607211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:14.607211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:14.607211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:14.607211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:14.607211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:14.607211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:14.607211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:14.607211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:14.607211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:14.607211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:14.607211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:14.607211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:14.607211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:14.607211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:14.608459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:14.608459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:14.608459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:14.608459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:14.608459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:14.608459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:14.608459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:14.608459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:14.608459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:14.608459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:14.608459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:14.608459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:14.608459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:14.608459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:14.608459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:14.608459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:14.608459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:14.609553 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:14.609553 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:14.609553 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:14.609553 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 291/586] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:14.609805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b419601-badb-4dd3-9416-4461d3fc678c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:14.649934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c20c57-a798-4e5c-8cb6-be1b43b034e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:14.650791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c20c57-a798-4e5c-8cb6-be1b43b034e6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:14.651079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c20c57-a798-4e5c-8cb6-be1b43b034e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:14.651353 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c20c57-a798-4e5c-8cb6-be1b43b034e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:14.662405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c20c57-a798-4e5c-8cb6-be1b43b034e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:14.665045 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c20c57-a798-4e5c-8cb6-be1b43b034e6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:14.672206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:14.680523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-23c20c57-a798-4e5c-8cb6-be1b43b034e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:14.680523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:14.680523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:14.680523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:14.680523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:14.680523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:14.680523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:14.680523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:14.680523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:14.680523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:14.680523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:14.680523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:14.680523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:14.680523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:14.680523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:14.680523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:14.690942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:14.690942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:14.690942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:14.690942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:14.690942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:14.690942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:14.690942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:14.690942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:14.690942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:14.690942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:14.690942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:14.690942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:14.690942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:14.690942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:14.690942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:14.690942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:14.690942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:14.692093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:14.692093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:14.692093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:14.692093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 296/587] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:14.692093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c20c57-a798-4e5c-8cb6-be1b43b034e6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:14.738939 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b419601-badb-4dd3-9416-4461d3fc678c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:14.759155 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:14.779767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9cee5-3b86-4b12-bafd-b444947a9848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:14.780927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9cee5-3b86-4b12-bafd-b444947a9848 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:14.780927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9cee5-3b86-4b12-bafd-b444947a9848 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:14.781396 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9cee5-3b86-4b12-bafd-b444947a9848 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:14.782173 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f9cee5-3b86-4b12-bafd-b444947a9848 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:14.782590 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f9cee5-3b86-4b12-bafd-b444947a9848 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:14.785216 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f9cee5-3b86-4b12-bafd-b444947a9848 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:14.804333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 292/588] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:14.804333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9cee5-3b86-4b12-bafd-b444947a9848 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:14.818590 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c20c57-a798-4e5c-8cb6-be1b43b034e6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:14.863867 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f270dfa-8e7e-4409-bf07-7a5e45c5e01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:14.865099 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f270dfa-8e7e-4409-bf07-7a5e45c5e01d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:14.865556 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f270dfa-8e7e-4409-bf07-7a5e45c5e01d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:14.866257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f270dfa-8e7e-4409-bf07-7a5e45c5e01d None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:14.867054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f270dfa-8e7e-4409-bf07-7a5e45c5e01d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:14.867660 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f270dfa-8e7e-4409-bf07-7a5e45c5e01d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:14.869517 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f270dfa-8e7e-4409-bf07-7a5e45c5e01d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:14.886006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 297/589] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:14.887659 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f270dfa-8e7e-4409-bf07-7a5e45c5e01d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:14.928931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9cee5-3b86-4b12-bafd-b444947a9848 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:14.954208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:14.968509 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944da79a-33e9-46fc-8472-4fa72c50b4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:14.969635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944da79a-33e9-46fc-8472-4fa72c50b4fb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:14.970177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944da79a-33e9-46fc-8472-4fa72c50b4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:14.971439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944da79a-33e9-46fc-8472-4fa72c50b4fb None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:14.972290 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944da79a-33e9-46fc-8472-4fa72c50b4fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:14.972795 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944da79a-33e9-46fc-8472-4fa72c50b4fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:14.974408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944da79a-33e9-46fc-8472-4fa72c50b4fb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:14.994162 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 293/590] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:14.995669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944da79a-33e9-46fc-8472-4fa72c50b4fb None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:15.007427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f270dfa-8e7e-4409-bf07-7a5e45c5e01d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:15.045470 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:15.052100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71502e6-6dbb-4b0e-8111-e1ade1b14ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:15.056642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71502e6-6dbb-4b0e-8111-e1ade1b14ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:15.056642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71502e6-6dbb-4b0e-8111-e1ade1b14ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:15.056642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71502e6-6dbb-4b0e-8111-e1ade1b14ee6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:15.061732 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71502e6-6dbb-4b0e-8111-e1ade1b14ee6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:15.063444 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71502e6-6dbb-4b0e-8111-e1ade1b14ee6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:15.076427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-a71502e6-6dbb-4b0e-8111-e1ade1b14ee6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 11:46:15.076427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:15.076427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:15.076427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:15.076427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:15.076427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:15.076427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:15.076427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:15.076427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:15.076427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:15.076427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:15.076427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:15.076427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:15.076427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:15.076427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:15.076427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:15.077351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:15.077351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:15.077351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:15.077351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:15.077351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:15.077351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:15.077351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:15.077351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:15.077351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:15.077351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:15.077351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:15.077351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:15.077351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:15.077351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:15.077351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:15.077351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:15.077351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:15.077351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:15.078722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:15.078722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:15.078722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:15.078722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:15.078722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:15.078722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:15.078722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:15.078722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:15.078722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 11:46:15.078722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:15.081832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 298/591] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:46:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:15.082930 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71502e6-6dbb-4b0e-8111-e1ade1b14ee6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:15.125186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944da79a-33e9-46fc-8472-4fa72c50b4fb None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:15.165201 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f0918-86b4-4416-8d60-633494fb5e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:15.165839 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f0918-86b4-4416-8d60-633494fb5e26 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:15.166159 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f0918-86b4-4416-8d60-633494fb5e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:15.166431 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f0918-86b4-4416-8d60-633494fb5e26 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:15.175244 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f0918-86b4-4416-8d60-633494fb5e26 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:15.176958 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f0918-86b4-4416-8d60-633494fb5e26 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:15.193195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-021f0918-86b4-4416-8d60-633494fb5e26 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 11:46:15.193195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:15.193195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:15.193195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:15.193195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:15.193195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:15.193195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:15.193195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:15.193195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:15.193195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:15.193195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:15.193195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:15.193195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:15.193195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:15.193195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:15.193195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:15.194228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:15.194228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:15.194228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:15.194228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:15.194228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:15.194228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:15.194228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:15.194228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:15.194228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:15.194228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:15.194228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:15.194228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:15.194228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:15.194228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:15.194228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:15.194228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:15.194228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:15.194228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:15.195180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:15.195180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:15.195180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:15.195180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:15.195180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:15.195180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:15.195180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:15.195180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:15.195180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 11:46:15.195180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:15.195180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 294/592] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:15.195180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f0918-86b4-4416-8d60-633494fb5e26 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:15.214223 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71502e6-6dbb-4b0e-8111-e1ade1b14ee6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:15.236855 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:15.265352 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978283d8-299c-4481-8808-34cddc5bfd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:15.266025 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978283d8-299c-4481-8808-34cddc5bfd68 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:15.266296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978283d8-299c-4481-8808-34cddc5bfd68 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:15.266575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978283d8-299c-4481-8808-34cddc5bfd68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:15.273618 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978283d8-299c-4481-8808-34cddc5bfd68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:15.275095 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978283d8-299c-4481-8808-34cddc5bfd68 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:15.288445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-978283d8-299c-4481-8808-34cddc5bfd68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:15.288445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:15.288445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:15.288445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:15.288445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:15.288445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:15.288445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:15.288445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:15.288445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:15.288445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:15.288445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:15.288445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:15.288445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:15.288445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:15.288445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:15.289600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:15.289600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:15.289600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:15.289600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:15.289600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:15.289600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:15.289600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:15.289600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:15.289600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:15.289600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:15.289600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:15.289600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:15.289600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:15.289600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:15.289600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:15.289600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:15.289600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:15.290754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:15.290754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:15.290754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:15.290754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:15.290754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:15.290754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:15.290754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:15.290754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:15.298593 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 299/593] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:15.300137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978283d8-299c-4481-8808-34cddc5bfd68 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:15.321919 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:15.338147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f0918-86b4-4416-8d60-633494fb5e26 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:15.381938 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fb5eb-a7b2-49b2-939c-684692c04cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:15.383646 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fb5eb-a7b2-49b2-939c-684692c04cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:15.384179 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fb5eb-a7b2-49b2-939c-684692c04cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:15.384624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fb5eb-a7b2-49b2-939c-684692c04cf7 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:15.385685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83fb5eb-a7b2-49b2-939c-684692c04cf7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:15.386159 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83fb5eb-a7b2-49b2-939c-684692c04cf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:15.388985 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83fb5eb-a7b2-49b2-939c-684692c04cf7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:15.411193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978283d8-299c-4481-8808-34cddc5bfd68 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:15.413372 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 295/594] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Sep 14 11:46:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:15.414738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fb5eb-a7b2-49b2-939c-684692c04cf7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:15.456157 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e1058-4905-4036-91bd-09dcaf678417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:15.457057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e1058-4905-4036-91bd-09dcaf678417 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:15.458037 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e1058-4905-4036-91bd-09dcaf678417 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:15.458476 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e1058-4905-4036-91bd-09dcaf678417 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:15.459375 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e1058-4905-4036-91bd-09dcaf678417 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:15.460297 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e1058-4905-4036-91bd-09dcaf678417 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:15.462668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e1058-4905-4036-91bd-09dcaf678417 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:15.485720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 300/595] 10.222.0.62 () {56 vars in 1139 bytes} [Wed Sep 14 11:46:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:15.486733 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e1058-4905-4036-91bd-09dcaf678417 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:15.529589 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fb5eb-a7b2-49b2-939c-684692c04cf7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:15.530871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:15.571719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d1a56-3f7d-4beb-8575-2a1220370709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:15.572720 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d1a56-3f7d-4beb-8575-2a1220370709 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:15.573299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d1a56-3f7d-4beb-8575-2a1220370709 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:15.573803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d1a56-3f7d-4beb-8575-2a1220370709 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:15.574843 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d1a56-3f7d-4beb-8575-2a1220370709 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:15.575865 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d1a56-3f7d-4beb-8575-2a1220370709 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:15.577499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d1a56-3f7d-4beb-8575-2a1220370709 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:15.595786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 296/596] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:15.597023 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d1a56-3f7d-4beb-8575-2a1220370709 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:15.614716 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:15.616598 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e1058-4905-4036-91bd-09dcaf678417 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:15.664465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312b6ce-fc8e-4428-9978-ef5c23d96932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:15.665602 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312b6ce-fc8e-4428-9978-ef5c23d96932 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:15.666082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312b6ce-fc8e-4428-9978-ef5c23d96932 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:15.666504 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312b6ce-fc8e-4428-9978-ef5c23d96932 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:15.679935 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312b6ce-fc8e-4428-9978-ef5c23d96932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:15.682485 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312b6ce-fc8e-4428-9978-ef5c23d96932 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:15.703475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2312b6ce-fc8e-4428-9978-ef5c23d96932 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:15.703475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:15.703475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:15.703475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:15.703475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:15.703475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:15.703475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:15.703475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:15.703475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:15.703475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:15.703475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:15.703475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:15.703475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:15.703475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:15.703475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:15.703475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:15.704630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:15.704630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:15.704630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:15.704630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:15.704630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:15.704630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:15.704630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:15.704630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:15.704630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:15.704630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:15.704630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:15.704630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:15.704630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:15.704630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:15.704630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:15.704630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:15.704630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:15.705724 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:15.705724 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:15.705724 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:15.706916 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d1a56-3f7d-4beb-8575-2a1220370709 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:15.707575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 301/597] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:15.708533 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312b6ce-fc8e-4428-9978-ef5c23d96932 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:15.747816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc03362-566f-485a-87c0-5f3ce01a7274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:15.748581 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc03362-566f-485a-87c0-5f3ce01a7274 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:15.748962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc03362-566f-485a-87c0-5f3ce01a7274 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:15.749357 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc03362-566f-485a-87c0-5f3ce01a7274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:15.760504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc03362-566f-485a-87c0-5f3ce01a7274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:15.762193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc03362-566f-485a-87c0-5f3ce01a7274 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:15.777617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-cdc03362-566f-485a-87c0-5f3ce01a7274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:15.777617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:15.777617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:15.777617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:15.777617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:15.777617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:15.777617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:15.777617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:15.777617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:15.777617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:15.777617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:15.777617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:15.777617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:15.777617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:15.777617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:15.777617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:15.778890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:15.778890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:15.778890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:15.778890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:15.778890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:15.778890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:15.778890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:15.778890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:15.778890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:15.778890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:15.778890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:15.778890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:15.778890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:15.778890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:15.778890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:15.778890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:15.778890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:15.779981 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:15.779981 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:15.779981 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:15.785460 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 297/598] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:15.786439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc03362-566f-485a-87c0-5f3ce01a7274 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:15.822557 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:15.846822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312b6ce-fc8e-4428-9978-ef5c23d96932 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:15.868028 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:15.893232 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b2d21-853d-4698-93b1-eb98e89662a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:15.893913 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b2d21-853d-4698-93b1-eb98e89662a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:15.894202 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b2d21-853d-4698-93b1-eb98e89662a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:15.894524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b2d21-853d-4698-93b1-eb98e89662a2 None admin] PATH_INFO: `/v3/projects/449a5fac1acd478abea544446c08ecd9/users/4b05d0e65e9645f3810a6d96c8fed9fc/roles/d1bb55f25dff4694b8675e0ef5fc4ec7` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:15.909317 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc03362-566f-485a-87c0-5f3ce01a7274 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:15.911653 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b2d21-853d-4698-93b1-eb98e89662a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=449a5fac1acd478abea544446c08ecd9, user_id=4b05d0e65e9645f3810a6d96c8fed9fc, role_id=d1bb55f25dff4694b8675e0ef5fc4ec7)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:15.913255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b2d21-853d-4698-93b1-eb98e89662a2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:15.946718 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e7cf2-e192-4c3f-9f72-1ab9b485285a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:15.947638 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e7cf2-e192-4c3f-9f72-1ab9b485285a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:15.948108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e7cf2-e192-4c3f-9f72-1ab9b485285a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:15.948539 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e7cf2-e192-4c3f-9f72-1ab9b485285a None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:15.949382 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e7cf2-e192-4c3f-9f72-1ab9b485285a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:15.949845 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e7cf2-e192-4c3f-9f72-1ab9b485285a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:15.952040 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e7cf2-e192-4c3f-9f72-1ab9b485285a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:15.964194 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 298/599] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:15.965228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e7cf2-e192-4c3f-9f72-1ab9b485285a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:15.981829 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 302/600] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:46:15 2022] PUT /identity/v3/projects/449a5fac1acd478abea544446c08ecd9/users/4b05d0e65e9645f3810a6d96c8fed9fc/roles/d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:46:15.982885 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b2d21-853d-4698-93b1-eb98e89662a2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:16.079862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e7cf2-e192-4c3f-9f72-1ab9b485285a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:16.109420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:16.142832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7057e5-c8ca-4a01-8522-751fc1253648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:16.143466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7057e5-c8ca-4a01-8522-751fc1253648 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:16.143809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7057e5-c8ca-4a01-8522-751fc1253648 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:16.144088 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7057e5-c8ca-4a01-8522-751fc1253648 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:16.144741 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7057e5-c8ca-4a01-8522-751fc1253648 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:16.145044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7057e5-c8ca-4a01-8522-751fc1253648 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:16.146538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7057e5-c8ca-4a01-8522-751fc1253648 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:16.177580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 299/601] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:16.185317 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b2d21-853d-4698-93b1-eb98e89662a2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:16.188417 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7057e5-c8ca-4a01-8522-751fc1253648 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:16.271564 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af8365-43d8-419e-a39b-76b4fcaf1822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:16.272980 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af8365-43d8-419e-a39b-76b4fcaf1822 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:16.273461 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af8365-43d8-419e-a39b-76b4fcaf1822 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:16.274056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af8365-43d8-419e-a39b-76b4fcaf1822 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:16.277802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:16.284717 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af8365-43d8-419e-a39b-76b4fcaf1822 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:16.286550 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af8365-43d8-419e-a39b-76b4fcaf1822 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:16.300528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-59af8365-43d8-419e-a39b-76b4fcaf1822 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:16.300528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:16.300528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:16.300528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:16.300528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:16.300528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:16.300528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:16.300528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:16.300528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:16.300528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:16.300528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:16.300528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:16.300528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:16.300528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:16.300528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:16.311961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:16.311961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:16.311961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:16.311961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:16.311961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:16.311961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:16.311961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:16.311961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:16.311961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:16.311961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:16.311961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:16.311961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:16.311961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:16.311961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:16.311961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:16.311961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:16.311961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:16.312851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:16.312851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:16.312851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:16.312851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:16.312851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:16.312851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:16.312851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:16.312851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:16.312851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 303/602] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:16.313338 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af8365-43d8-419e-a39b-76b4fcaf1822 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:16.336585 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7057e5-c8ca-4a01-8522-751fc1253648 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:16.380622 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d142d06-8427-4ce7-80ad-dc6ad620731f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:16.381746 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d142d06-8427-4ce7-80ad-dc6ad620731f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:16.382253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d142d06-8427-4ce7-80ad-dc6ad620731f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:16.383176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d142d06-8427-4ce7-80ad-dc6ad620731f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:16.396816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d142d06-8427-4ce7-80ad-dc6ad620731f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:16.398572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d142d06-8427-4ce7-80ad-dc6ad620731f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:16.421881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-8d142d06-8427-4ce7-80ad-dc6ad620731f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:16.421881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:16.421881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:16.421881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:16.421881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:16.421881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:16.421881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:16.421881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:16.421881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:16.421881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:16.421881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:16.421881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:16.421881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:16.421881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:16.421881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:16.422819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:16.422819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:16.422819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:16.422819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:16.422819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:16.422819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:16.422819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:16.422819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:16.422819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:16.422819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:16.422819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:16.422819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:16.422819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:16.422819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:16.422819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:16.422819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:16.422819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:16.423719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:16.423719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:16.423719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:16.423719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:16.423719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:16.423719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:16.423719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:16.423719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:16.424147 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af8365-43d8-419e-a39b-76b4fcaf1822 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:16.427521 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 300/603] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:16.432640 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d142d06-8427-4ce7-80ad-dc6ad620731f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:16.467163 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd6e40-6e45-477e-8643-7a17751eb30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:16.467664 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd6e40-6e45-477e-8643-7a17751eb30a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:16.467947 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd6e40-6e45-477e-8643-7a17751eb30a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:16.468219 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd6e40-6e45-477e-8643-7a17751eb30a None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:16.469038 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd6e40-6e45-477e-8643-7a17751eb30a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:16.469280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd6e40-6e45-477e-8643-7a17751eb30a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:16.470753 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd6e40-6e45-477e-8643-7a17751eb30a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:16.485676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:16.496090 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 304/604] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:16.498136 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd6e40-6e45-477e-8643-7a17751eb30a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:16.547092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d142d06-8427-4ce7-80ad-dc6ad620731f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:16.585098 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bba20-b43f-4399-b797-f8c94ab45b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:16.586267 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bba20-b43f-4399-b797-f8c94ab45b38 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:16.586781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bba20-b43f-4399-b797-f8c94ab45b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:16.587275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bba20-b43f-4399-b797-f8c94ab45b38 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:16.588164 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bba20-b43f-4399-b797-f8c94ab45b38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:16.588671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bba20-b43f-4399-b797-f8c94ab45b38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:16.590369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bba20-b43f-4399-b797-f8c94ab45b38 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:16.604108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 301/605] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:46:16.610159 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bba20-b43f-4399-b797-f8c94ab45b38 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:16.647982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:16.711613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd6e40-6e45-477e-8643-7a17751eb30a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:16.749635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf5066-90ca-4e25-88fb-4e8b06741460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:16.750285 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf5066-90ca-4e25-88fb-4e8b06741460 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:16.750568 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf5066-90ca-4e25-88fb-4e8b06741460 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:16.750844 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf5066-90ca-4e25-88fb-4e8b06741460 None admin] PATH_INFO: `/v3/projects/449a5fac1acd478abea544446c08ecd9/users/d27c7ff726e443e89da94f282b5cddab/roles/d1bb55f25dff4694b8675e0ef5fc4ec7` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:16.759089 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bba20-b43f-4399-b797-f8c94ab45b38 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:16.767470 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:16.769087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf5066-90ca-4e25-88fb-4e8b06741460 None admin] RBAC: Authorizing `identity:create_grant(project_id=449a5fac1acd478abea544446c08ecd9, user_id=d27c7ff726e443e89da94f282b5cddab, role_id=d1bb55f25dff4694b8675e0ef5fc4ec7)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:16.770582 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf5066-90ca-4e25-88fb-4e8b06741460 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:16.794895 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d5b6f-5f14-49a9-9f54-bb51d2e2e50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:16.795543 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d5b6f-5f14-49a9-9f54-bb51d2e2e50a None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:16.795820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d5b6f-5f14-49a9-9f54-bb51d2e2e50a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:16.796089 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d5b6f-5f14-49a9-9f54-bb51d2e2e50a None admin] PATH_INFO: `/v3/projects/449a5fac1acd478abea544446c08ecd9/users/13f1840d92d94c3ebd16000a4889c377/roles/d1bb55f25dff4694b8675e0ef5fc4ec7` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:16.810740 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d5b6f-5f14-49a9-9f54-bb51d2e2e50a None admin] RBAC: Authorizing `identity:create_grant(project_id=449a5fac1acd478abea544446c08ecd9, user_id=13f1840d92d94c3ebd16000a4889c377, role_id=d1bb55f25dff4694b8675e0ef5fc4ec7)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:16.812213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d5b6f-5f14-49a9-9f54-bb51d2e2e50a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:16.854289 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 305/606] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:46:16 2022] PUT /identity/v3/projects/449a5fac1acd478abea544446c08ecd9/users/d27c7ff726e443e89da94f282b5cddab/roles/d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 11:46:16.855542 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9b386-4350-44a2-9657-a19796d32b5d None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:16.855835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9b386-4350-44a2-9657-a19796d32b5d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:16.856109 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9b386-4350-44a2-9657-a19796d32b5d None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:16.860784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 306/607] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:16.860784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c2caa-0217-4a9b-b81f-fd99d6c041f4 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:16.860784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c2caa-0217-4a9b-b81f-fd99d6c041f4 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:16.860784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c2caa-0217-4a9b-b81f-fd99d6c041f4 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:16.860784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 307/608] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:16.865992 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487580d3-4f25-496c-b57a-6b5b12b775d1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:16.865992 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487580d3-4f25-496c-b57a-6b5b12b775d1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:16.865992 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487580d3-4f25-496c-b57a-6b5b12b775d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:16.889901 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 302/609] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:46:16 2022] PUT /identity/v3/projects/449a5fac1acd478abea544446c08ecd9/users/13f1840d92d94c3ebd16000a4889c377/roles/d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:46:16.889901 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6e405-8deb-4017-a57e-01738a80ccd1 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:16.889901 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6e405-8deb-4017-a57e-01738a80ccd1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:16.889901 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6e405-8deb-4017-a57e-01738a80ccd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:16.911329 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-487580d3-4f25-496c-b57a-6b5b12b775d1 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:16.914774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-487580d3-4f25-496c-b57a-6b5b12b775d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:16.928844 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-43a6e405-8deb-4017-a57e-01738a80ccd1 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:16.932718 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-43a6e405-8deb-4017-a57e-01738a80ccd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:17.067611 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 308/610] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:17.071189 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 303/611] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:17.354071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487580d3-4f25-496c-b57a-6b5b12b775d1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:17.372586 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6e405-8deb-4017-a57e-01738a80ccd1 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:17.414253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:17.504243 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487580d3-4f25-496c-b57a-6b5b12b775d1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:17.526202 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:17.542163 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afecf9a-0d7c-491d-b8c8-dbc8770036a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:17.542799 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afecf9a-0d7c-491d-b8c8-dbc8770036a9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:17.543082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afecf9a-0d7c-491d-b8c8-dbc8770036a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:17.543356 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afecf9a-0d7c-491d-b8c8-dbc8770036a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:17.554120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6e405-8deb-4017-a57e-01738a80ccd1 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:17.569726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afecf9a-0d7c-491d-b8c8-dbc8770036a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:17.574677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-6afecf9a-0d7c-491d-b8c8-dbc8770036a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:17.574677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:17.574677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:17.574677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:17.574677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:17.574677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:17.574677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:17.574677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:17.574677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:17.574677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:17.574677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:17.574677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:17.574677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:17.574677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:17.574677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:17.574677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:17.576227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:17.576227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:17.576227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:17.576227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:17.576227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:17.576227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:17.576227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:17.576227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:17.576227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:17.576227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:17.576227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:17.576227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:17.576227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:17.576227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:17.576227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:17.576227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:17.577066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:17.577066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:17.577066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:17.577066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:17.577066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:17.577066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:17.577066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:17.577066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:17.577066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:17.577066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:17.577066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:17.577066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:17.577066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 309/612] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:17.582519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afecf9a-0d7c-491d-b8c8-dbc8770036a9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:17.635342 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498bff3-0526-48b9-9e70-d6646b6f158f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:17.636708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498bff3-0526-48b9-9e70-d6646b6f158f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:17.637410 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498bff3-0526-48b9-9e70-d6646b6f158f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:17.638412 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498bff3-0526-48b9-9e70-d6646b6f158f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:17.661979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498bff3-0526-48b9-9e70-d6646b6f158f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:17.674178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-3498bff3-0526-48b9-9e70-d6646b6f158f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:17.674178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:17.674178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:17.674178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:17.674178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:17.674178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:17.674178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:17.674178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:17.674178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:17.674178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:17.674178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:17.674178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:17.674178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:17.674178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:17.674178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:17.674178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:17.675369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:17.675369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:17.675369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:17.675369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:17.675369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:17.675369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:17.675369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:17.675369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:17.675369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:17.675369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:17.675369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:17.675369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:17.675369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:17.675369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:17.675369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:17.675369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:17.676204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:17.676204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:17.676204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:17.676204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:17.676204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:17.676204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:17.676204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:17.676204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:17.676204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:17.676204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:17.676204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:17.676204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:17.677565 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 304/613] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:17.681106 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498bff3-0526-48b9-9e70-d6646b6f158f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:17.702758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:17.708447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afecf9a-0d7c-491d-b8c8-dbc8770036a9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:17.752392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b072969-9687-4330-b50b-6301c4955c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:17.754120 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b072969-9687-4330-b50b-6301c4955c7b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:17.754405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b072969-9687-4330-b50b-6301c4955c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:17.754678 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b072969-9687-4330-b50b-6301c4955c7b None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:17.755352 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b072969-9687-4330-b50b-6301c4955c7b None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:17.757924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b072969-9687-4330-b50b-6301c4955c7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:17.759346 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b072969-9687-4330-b50b-6301c4955c7b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:17.781515 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 310/614] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:46:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:46:17.787264 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b072969-9687-4330-b50b-6301c4955c7b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:17.911215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498bff3-0526-48b9-9e70-d6646b6f158f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:17.966439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb1419-0592-44ab-b6b9-a4327656b31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:17.967119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb1419-0592-44ab-b6b9-a4327656b31c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:17.967447 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb1419-0592-44ab-b6b9-a4327656b31c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:17.967729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb1419-0592-44ab-b6b9-a4327656b31c None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:17.968397 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb1419-0592-44ab-b6b9-a4327656b31c None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:17.968703 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb1419-0592-44ab-b6b9-a4327656b31c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:17.970103 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb1419-0592-44ab-b6b9-a4327656b31c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:17.979147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 305/615] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:46:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:17.984541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb1419-0592-44ab-b6b9-a4327656b31c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:17.997571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b072969-9687-4330-b50b-6301c4955c7b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:18.005669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:18.063464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fec15c-48cc-4196-84a5-3059a87baabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:18.064118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fec15c-48cc-4196-84a5-3059a87baabe None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:18.064400 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fec15c-48cc-4196-84a5-3059a87baabe None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:18.064677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fec15c-48cc-4196-84a5-3059a87baabe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:18.081407 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fec15c-48cc-4196-84a5-3059a87baabe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:18.083210 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fec15c-48cc-4196-84a5-3059a87baabe None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:18.103537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb1419-0592-44ab-b6b9-a4327656b31c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:18.104768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-b3fec15c-48cc-4196-84a5-3059a87baabe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:18.104768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:18.104768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:18.104768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:18.104768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:18.104768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:18.104768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:18.104768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:18.104768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:18.104768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:18.104768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:18.104768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:18.104768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:18.104768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:18.104768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:18.104768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:18.105702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:18.105702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:18.105702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:18.105702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:18.105702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:18.105702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:18.105702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:18.105702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:18.105702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:18.105702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:18.105702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:18.105702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:18.105702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:18.105702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:18.105702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:18.105702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:18.105702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:18.106684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:18.106684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:18.106684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:18.114237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 311/616] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:18.115838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fec15c-48cc-4196-84a5-3059a87baabe None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:18.140867 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:18.143189 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4f8bb-86d2-46c3-8cbf-8c536b50e8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:18.143847 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4f8bb-86d2-46c3-8cbf-8c536b50e8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:18.144129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4f8bb-86d2-46c3-8cbf-8c536b50e8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:18.144404 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4f8bb-86d2-46c3-8cbf-8c536b50e8c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:18.168565 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4f8bb-86d2-46c3-8cbf-8c536b50e8c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:18.169998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4f8bb-86d2-46c3-8cbf-8c536b50e8c3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:18.201823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-dad4f8bb-86d2-46c3-8cbf-8c536b50e8c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:18.201823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:18.201823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:18.201823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:18.201823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:18.201823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:18.201823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:18.201823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:18.201823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:18.201823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:18.201823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:18.201823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:18.201823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:18.201823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:18.201823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:18.201823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:18.202864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:18.202864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:18.202864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:18.202864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:18.202864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:18.202864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:18.202864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:18.202864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:18.202864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:18.202864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:18.202864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:18.202864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:18.202864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:18.202864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:18.202864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:18.202864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:18.202864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:18.203774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:18.203774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:18.203774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:18.203774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 306/617] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:18.209215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4f8bb-86d2-46c3-8cbf-8c536b50e8c3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:18.293042 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fec15c-48cc-4196-84a5-3059a87baabe None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:18.334076 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:18.361111 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eeffe6-7384-44b4-9339-65212ff2954d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:18.361743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eeffe6-7384-44b4-9339-65212ff2954d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:18.363743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eeffe6-7384-44b4-9339-65212ff2954d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:18.363743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eeffe6-7384-44b4-9339-65212ff2954d None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:18.363743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eeffe6-7384-44b4-9339-65212ff2954d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:18.363743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eeffe6-7384-44b4-9339-65212ff2954d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:18.364695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eeffe6-7384-44b4-9339-65212ff2954d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:18.379467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 312/618] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:46:18.390194 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eeffe6-7384-44b4-9339-65212ff2954d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:18.404333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4f8bb-86d2-46c3-8cbf-8c536b50e8c3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:18.470735 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba7fa7-117d-4c43-94c8-f7321a54927f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:18.472333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba7fa7-117d-4c43-94c8-f7321a54927f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:18.472615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba7fa7-117d-4c43-94c8-f7321a54927f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:18.472888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba7fa7-117d-4c43-94c8-f7321a54927f None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:18.473534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba7fa7-117d-4c43-94c8-f7321a54927f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:18.473839 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba7fa7-117d-4c43-94c8-f7321a54927f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:18.478284 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba7fa7-117d-4c43-94c8-f7321a54927f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:18.503231 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 307/619] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:46:18.509477 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba7fa7-117d-4c43-94c8-f7321a54927f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:18.523338 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:18.621626 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba7fa7-117d-4c43-94c8-f7321a54927f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:18.624370 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eeffe6-7384-44b4-9339-65212ff2954d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:18.646289 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:18.659620 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c0afe-d741-4dfa-97bb-a07ab0efac86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:18.660354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c0afe-d741-4dfa-97bb-a07ab0efac86 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:18.660728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c0afe-d741-4dfa-97bb-a07ab0efac86 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:18.661302 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c0afe-d741-4dfa-97bb-a07ab0efac86 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:18.669005 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c0afe-d741-4dfa-97bb-a07ab0efac86 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:18.670625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c0afe-d741-4dfa-97bb-a07ab0efac86 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:18.683963 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362eda8-8018-46da-a1fd-939700753d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:18.684299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-779c0afe-d741-4dfa-97bb-a07ab0efac86 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 11:46:18.684299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:18.684299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:18.684299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:18.684299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:18.684299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:18.684299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:18.684299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:18.684299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:18.684299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:18.684299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:18.684299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:18.684299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:18.684299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:18.684299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:18.684299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:18.685255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:18.685255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:18.685255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:18.685255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:18.685255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:18.685255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:18.685255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:18.685255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:18.685255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:18.685255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:18.685255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:18.685255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:18.685255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:18.685255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:18.685255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:18.685255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:18.685255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:18.685255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:18.686467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:18.686467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:18.686467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:18.686467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:18.686467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:18.686467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:18.686467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:18.686467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:18.686467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 11:46:18.686467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:18.686467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 308/620] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:46:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:18.688062 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362eda8-8018-46da-a1fd-939700753d46 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:18.688515 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362eda8-8018-46da-a1fd-939700753d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:18.688954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362eda8-8018-46da-a1fd-939700753d46 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:18.691637 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e64953-a273-4cb7-9700-c21cc3f10a11 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:18.691993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e64953-a273-4cb7-9700-c21cc3f10a11 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:18.692349 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e64953-a273-4cb7-9700-c21cc3f10a11 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:18.697894 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 309/621] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:18.697894 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e64953-a273-4cb7-9700-c21cc3f10a11 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:18.703144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362eda8-8018-46da-a1fd-939700753d46 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:18.704775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362eda8-8018-46da-a1fd-939700753d46 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:18.717635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-6362eda8-8018-46da-a1fd-939700753d46 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 11:46:18.717635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:18.717635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:18.717635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:18.717635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:18.717635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:18.717635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:18.717635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:18.717635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:18.717635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:18.717635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:18.717635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:18.717635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:18.717635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:18.717635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:18.717635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:18.718713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:18.718713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:18.718713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:18.718713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:18.718713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:18.718713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:18.718713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:18.718713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:18.718713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:18.718713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:18.718713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:18.718713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:18.718713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:18.718713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:18.718713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:18.718713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:18.718713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:18.718713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:18.719689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:18.719689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:18.719689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:18.719689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:18.719689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:18.719689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:18.719689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:18.719689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:18.719689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 11:46:18.719689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:18.722270 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 313/622] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Sep 14 11:46:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:18.723701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c4839-47fe-4ebf-9d14-47e6fb0f0d0c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:18.724160 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c4839-47fe-4ebf-9d14-47e6fb0f0d0c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:18.724595 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c4839-47fe-4ebf-9d14-47e6fb0f0d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:18.743115 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e61c4839-47fe-4ebf-9d14-47e6fb0f0d0c None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:18.744633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e61c4839-47fe-4ebf-9d14-47e6fb0f0d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:18.818278 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 314/623] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:18.820570 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c4839-47fe-4ebf-9d14-47e6fb0f0d0c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:18.837133 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e64953-a273-4cb7-9700-c21cc3f10a11 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:18.900952 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebd6de-69b8-41d8-ac85-0b8d31784402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:18.901610 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebd6de-69b8-41d8-ac85-0b8d31784402 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:18.901911 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebd6de-69b8-41d8-ac85-0b8d31784402 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:18.902241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebd6de-69b8-41d8-ac85-0b8d31784402 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:18.902982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebd6de-69b8-41d8-ac85-0b8d31784402 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:18.903375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebd6de-69b8-41d8-ac85-0b8d31784402 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:18.904802 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebd6de-69b8-41d8-ac85-0b8d31784402 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:18.939466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 310/624] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Sep 14 11:46:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:18.941216 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:18.946782 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebd6de-69b8-41d8-ac85-0b8d31784402 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:19.015393 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c4839-47fe-4ebf-9d14-47e6fb0f0d0c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:19.020281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:19.083248 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0022bff-9747-4beb-98f1-34219e88877b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:19.083862 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0022bff-9747-4beb-98f1-34219e88877b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:19.084144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0022bff-9747-4beb-98f1-34219e88877b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:19.084418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0022bff-9747-4beb-98f1-34219e88877b None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:19.085155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0022bff-9747-4beb-98f1-34219e88877b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:19.085462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0022bff-9747-4beb-98f1-34219e88877b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:19.086986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0022bff-9747-4beb-98f1-34219e88877b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:19.119560 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 315/625] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Sep 14 11:46:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:19.126509 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0022bff-9747-4beb-98f1-34219e88877b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:19.137583 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebd6de-69b8-41d8-ac85-0b8d31784402 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:19.184538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109885c-1a84-4c15-9366-7c6cc782b1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:19.185750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109885c-1a84-4c15-9366-7c6cc782b1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:19.186078 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109885c-1a84-4c15-9366-7c6cc782b1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:19.186351 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109885c-1a84-4c15-9366-7c6cc782b1b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:19.201727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109885c-1a84-4c15-9366-7c6cc782b1b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:19.203837 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109885c-1a84-4c15-9366-7c6cc782b1b9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:19.224336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-7109885c-1a84-4c15-9366-7c6cc782b1b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:19.224336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:19.224336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:19.224336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:19.224336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:19.224336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:19.224336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:19.224336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:19.224336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:19.224336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:19.224336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:19.224336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:19.224336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:19.224336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:19.224336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:19.224336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:19.225369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:19.225369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:19.225369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:19.225369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:19.225369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:19.225369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:19.225369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:19.225369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:19.225369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:19.225369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:19.225369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:19.225369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:19.225369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:19.225369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:19.225369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:19.225369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:19.225369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:19.226295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:19.226295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:19.226295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:19.228801 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 311/626] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:19.229742 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109885c-1a84-4c15-9366-7c6cc782b1b9 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:19.295208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0022bff-9747-4beb-98f1-34219e88877b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:19.316825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:19.332980 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:19.335067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8e95e-19aa-4a8f-a122-ee146475ddd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:19.335067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8e95e-19aa-4a8f-a122-ee146475ddd4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:19.335067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8e95e-19aa-4a8f-a122-ee146475ddd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:19.335067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8e95e-19aa-4a8f-a122-ee146475ddd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:19.343853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109885c-1a84-4c15-9366-7c6cc782b1b9 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:19.388070 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea4540-5e2b-4376-a8c3-222353125f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:19.388720 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea4540-5e2b-4376-a8c3-222353125f89 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:19.388998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea4540-5e2b-4376-a8c3-222353125f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:19.389268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea4540-5e2b-4376-a8c3-222353125f89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:19.400197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea4540-5e2b-4376-a8c3-222353125f89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:19.405039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-85ea4540-5e2b-4376-a8c3-222353125f89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:19.405039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:19.405039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:19.405039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:19.405039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:19.405039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:19.405039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:19.405039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:19.405039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:19.405039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:19.405039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:19.405039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:19.405039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:19.405039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:19.405039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:19.405039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:19.405992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:19.405992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:19.405992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:19.405992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:19.405992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:19.405992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:19.405992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:19.405992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:19.405992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:19.405992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:19.405992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:19.405992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:19.405992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:19.405992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:19.405992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:19.405992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:19.406836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:19.406836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:19.406836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:19.406836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:19.406836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:19.406836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:19.406836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:19.406836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:19.406836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:19.406836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:19.406836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:19.406836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:19.411029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 312/627] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:19.411859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea4540-5e2b-4376-a8c3-222353125f89 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:19.417933 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8e95e-19aa-4a8f-a122-ee146475ddd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:19.419918 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8e95e-19aa-4a8f-a122-ee146475ddd4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:19.514271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea4540-5e2b-4376-a8c3-222353125f89 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:19.521126 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 316/628] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:46:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 11:46:19.522324 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d127116-b5d2-4c50-96e2-599ea7e9770f None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:19.522614 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d127116-b5d2-4c50-96e2-599ea7e9770f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:19.522895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d127116-b5d2-4c50-96e2-599ea7e9770f None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:19.524266 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 317/629] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:19.524818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d127116-b5d2-4c50-96e2-599ea7e9770f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:19.545004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:19.552323 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb43d86-72b5-48fa-9c91-4b1966bde6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:19.552323 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb43d86-72b5-48fa-9c91-4b1966bde6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:19.552323 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb43d86-72b5-48fa-9c91-4b1966bde6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:19.552323 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb43d86-72b5-48fa-9c91-4b1966bde6c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:19.552746 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb43d86-72b5-48fa-9c91-4b1966bde6c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:19.553056 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb43d86-72b5-48fa-9c91-4b1966bde6c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:19.554442 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb43d86-72b5-48fa-9c91-4b1966bde6c4 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:19.565994 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 313/630] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:19.567265 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2ffe2-208f-4a5d-8b10-f80dce6fd2bc None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:19.567544 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2ffe2-208f-4a5d-8b10-f80dce6fd2bc None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:19.567815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2ffe2-208f-4a5d-8b10-f80dce6fd2bc None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:19.570731 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 314/631] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:19.570731 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2ffe2-208f-4a5d-8b10-f80dce6fd2bc None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:19.581904 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:19.627703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d127116-b5d2-4c50-96e2-599ea7e9770f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:19.661009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999b4c2-9d5a-4e50-aac4-986db194c616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:19.661606 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999b4c2-9d5a-4e50-aac4-986db194c616 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:19.661906 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999b4c2-9d5a-4e50-aac4-986db194c616 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:19.662182 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999b4c2-9d5a-4e50-aac4-986db194c616 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:19.662843 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999b4c2-9d5a-4e50-aac4-986db194c616 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:19.663147 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999b4c2-9d5a-4e50-aac4-986db194c616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:19.664525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999b4c2-9d5a-4e50-aac4-986db194c616 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:19.670190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2ffe2-208f-4a5d-8b10-f80dce6fd2bc None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:19.672766 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 318/632] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:46:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:19.674085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66843a-77ef-4b15-b3c5-315a80f96900 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:19.674634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66843a-77ef-4b15-b3c5-315a80f96900 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:19.674908 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66843a-77ef-4b15-b3c5-315a80f96900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:19.693604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-7e66843a-77ef-4b15-b3c5-315a80f96900 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:19.694851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7e66843a-77ef-4b15-b3c5-315a80f96900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:19.706234 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2682b-1438-40ec-ab70-1f1a6809b447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:19.706860 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2682b-1438-40ec-ab70-1f1a6809b447 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:19.707208 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2682b-1438-40ec-ab70-1f1a6809b447 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:19.707489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2682b-1438-40ec-ab70-1f1a6809b447 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:19.717961 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf2682b-1438-40ec-ab70-1f1a6809b447 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:19.719423 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf2682b-1438-40ec-ab70-1f1a6809b447 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:19.734315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-dcf2682b-1438-40ec-ab70-1f1a6809b447 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:19.734315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:19.734315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:19.734315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:19.734315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:19.734315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:19.734315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:19.734315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:19.734315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:19.734315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:19.734315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:19.734315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:19.734315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:19.734315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:19.734315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:19.734315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:19.735322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:19.735322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:19.735322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:19.735322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:19.735322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:19.735322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:19.735322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:19.735322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:19.735322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:19.735322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:19.735322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:19.735322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:19.735322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:19.735322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:19.735322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:19.735322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:19.735322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:19.736209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:19.736209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:19.736209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:19.736209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 315/633] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:19.736209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2682b-1438-40ec-ab70-1f1a6809b447 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:19.763284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 319/634] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:19.764506 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cbb2f-84e4-43fd-b7e5-74c1eb8901e1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:19.764792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cbb2f-84e4-43fd-b7e5-74c1eb8901e1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:19.765065 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cbb2f-84e4-43fd-b7e5-74c1eb8901e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:19.783279 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-145cbb2f-84e4-43fd-b7e5-74c1eb8901e1 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:19.784506 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-145cbb2f-84e4-43fd-b7e5-74c1eb8901e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:19.787821 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:19.809494 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:19.865892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 320/635] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:19.865892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145cbb2f-84e4-43fd-b7e5-74c1eb8901e1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:19.870939 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2682b-1438-40ec-ab70-1f1a6809b447 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:19.914539 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61c432-1445-42a0-a9f5-48f15db6fcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:19.915176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61c432-1445-42a0-a9f5-48f15db6fcc1 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:19.915481 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61c432-1445-42a0-a9f5-48f15db6fcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:19.915986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61c432-1445-42a0-a9f5-48f15db6fcc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:19.924722 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61c432-1445-42a0-a9f5-48f15db6fcc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:19.926717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61c432-1445-42a0-a9f5-48f15db6fcc1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:19.944705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-cb61c432-1445-42a0-a9f5-48f15db6fcc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:19.944705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:19.944705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:19.944705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:19.944705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:19.944705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:19.944705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:19.944705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:19.944705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:19.944705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:19.944705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:19.944705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:19.944705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:19.944705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:19.944705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:19.945642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:19.945642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:19.945642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:19.945642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:19.945642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:19.945642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:19.945642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:19.945642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:19.945642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:19.945642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:19.945642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:19.945642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:19.945642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:19.945642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:19.945642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:19.945642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:19.945642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:19.952628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:19.952628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:19.952628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:19.952628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:19.952628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:19.952628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:19.952628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:19.952628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:19.952628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 316/636] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:19.952628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61c432-1445-42a0-a9f5-48f15db6fcc1 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:19.987396 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145cbb2f-84e4-43fd-b7e5-74c1eb8901e1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:20.035611 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088982e-8aa7-4620-977d-0f87cda10e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:20.036086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088982e-8aa7-4620-977d-0f87cda10e9b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:20.036371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088982e-8aa7-4620-977d-0f87cda10e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:20.036649 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088982e-8aa7-4620-977d-0f87cda10e9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:20.045534 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:20.047715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088982e-8aa7-4620-977d-0f87cda10e9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:20.049158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088982e-8aa7-4620-977d-0f87cda10e9b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:20.060399 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61c432-1445-42a0-a9f5-48f15db6fcc1 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:20.064078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-4088982e-8aa7-4620-977d-0f87cda10e9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:20.064078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:20.064078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:20.064078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:20.064078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:20.064078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:20.064078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:20.064078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:20.064078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:20.064078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:20.064078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:20.064078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:20.064078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:20.064078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:20.064078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:20.064078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:20.065011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:20.065011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:20.065011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:20.065011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:20.065011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:20.065011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:20.065011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:20.065011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:20.065011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:20.065011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:20.065011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:20.065011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:20.065011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:20.065011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:20.065011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:20.065011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:20.065011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:20.070790 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:20.070790 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:20.070790 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:20.070790 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 321/637] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:20.070790 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088982e-8aa7-4620-977d-0f87cda10e9b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:20.088043 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:20.100088 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3bfed-c183-494e-ae5e-6489cf3b0b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:20.100805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3bfed-c183-494e-ae5e-6489cf3b0b1d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:20.103602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3bfed-c183-494e-ae5e-6489cf3b0b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:20.103602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3bfed-c183-494e-ae5e-6489cf3b0b1d None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:20.103602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3bfed-c183-494e-ae5e-6489cf3b0b1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:20.103602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3bfed-c183-494e-ae5e-6489cf3b0b1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:20.103915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3bfed-c183-494e-ae5e-6489cf3b0b1d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:20.121539 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 317/638] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:20.121539 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3bfed-c183-494e-ae5e-6489cf3b0b1d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:20.177572 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088982e-8aa7-4620-977d-0f87cda10e9b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:20.210533 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979310f-5ba5-4fb9-92f9-15377f076ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:20.211130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979310f-5ba5-4fb9-92f9-15377f076ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:20.211407 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979310f-5ba5-4fb9-92f9-15377f076ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:20.211681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979310f-5ba5-4fb9-92f9-15377f076ed7 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:20.212360 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979310f-5ba5-4fb9-92f9-15377f076ed7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:20.212667 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979310f-5ba5-4fb9-92f9-15377f076ed7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:20.214130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979310f-5ba5-4fb9-92f9-15377f076ed7 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:20.220567 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3bfed-c183-494e-ae5e-6489cf3b0b1d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:20.226049 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 322/639] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:20.226831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979310f-5ba5-4fb9-92f9-15377f076ed7 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:20.257410 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e21f0a-78e2-404a-a7e7-18fe025cb5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:20.258062 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e21f0a-78e2-404a-a7e7-18fe025cb5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:20.258343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e21f0a-78e2-404a-a7e7-18fe025cb5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:20.258616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e21f0a-78e2-404a-a7e7-18fe025cb5e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:20.268974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e21f0a-78e2-404a-a7e7-18fe025cb5e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:20.273805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-86e21f0a-78e2-404a-a7e7-18fe025cb5e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:20.273805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:20.273805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:20.273805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:20.273805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:20.273805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:20.273805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:20.273805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:20.273805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:20.273805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:20.273805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:20.273805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:20.273805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:20.273805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:20.273805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:20.273805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:20.274941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:20.274941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:20.274941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:20.274941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:20.274941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:20.274941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:20.274941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:20.274941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:20.274941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:20.274941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:20.274941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:20.274941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:20.274941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:20.274941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:20.274941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:20.274941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:20.275785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:20.275785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:20.275785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:20.275785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:20.275785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:20.275785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:20.275785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:20.275785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:20.275785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:20.275785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:20.275785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:20.275785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:20.275785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 318/640] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:20.275785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e21f0a-78e2-404a-a7e7-18fe025cb5e1 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:20.281775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:20.321243 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:20.328784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979310f-5ba5-4fb9-92f9-15377f076ed7 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:20.362543 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c809c-6fc7-41fc-83bf-1b6f65ecf7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:20.363146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c809c-6fc7-41fc-83bf-1b6f65ecf7cc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:20.363425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c809c-6fc7-41fc-83bf-1b6f65ecf7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:20.363694 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c809c-6fc7-41fc-83bf-1b6f65ecf7cc None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:20.364323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c809c-6fc7-41fc-83bf-1b6f65ecf7cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:20.364620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c809c-6fc7-41fc-83bf-1b6f65ecf7cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:20.366073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c809c-6fc7-41fc-83bf-1b6f65ecf7cc None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:20.377315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 323/641] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:20.378115 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c809c-6fc7-41fc-83bf-1b6f65ecf7cc None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:20.378889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e21f0a-78e2-404a-a7e7-18fe025cb5e1 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:20.417970 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73adf6e-3bd4-42fa-81d3-fba17c188160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:20.418604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73adf6e-3bd4-42fa-81d3-fba17c188160 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:20.418911 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73adf6e-3bd4-42fa-81d3-fba17c188160 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:20.419186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73adf6e-3bd4-42fa-81d3-fba17c188160 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:20.426547 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73adf6e-3bd4-42fa-81d3-fba17c188160 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:20.428282 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73adf6e-3bd4-42fa-81d3-fba17c188160 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:20.440820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-f73adf6e-3bd4-42fa-81d3-fba17c188160 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:20.440820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:20.440820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:20.440820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:20.440820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:20.440820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:20.440820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:20.440820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:20.440820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:20.440820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:20.440820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:20.440820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:20.440820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:20.440820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:20.440820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:20.442271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:20.442271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:20.442271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:20.442271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:20.442271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:20.442271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:20.442271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:20.442271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:20.442271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:20.442271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:20.442271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:20.442271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:20.442271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:20.442271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:20.442271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:20.442271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:20.442271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:20.443541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:20.443541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:20.443541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:20.443541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:20.443541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:20.443541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:20.443541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:20.443541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:20.446000 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 319/642] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:20.446947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73adf6e-3bd4-42fa-81d3-fba17c188160 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:20.481550 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c809c-6fc7-41fc-83bf-1b6f65ecf7cc None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:20.493787 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:20.515568 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317a9b1-360c-4c46-81c1-09341a707be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:20.516203 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317a9b1-360c-4c46-81c1-09341a707be7 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:20.516490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317a9b1-360c-4c46-81c1-09341a707be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:20.516775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317a9b1-360c-4c46-81c1-09341a707be7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:20.529429 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317a9b1-360c-4c46-81c1-09341a707be7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:20.534380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-5317a9b1-360c-4c46-81c1-09341a707be7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:20.534380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:20.534380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:20.534380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:20.534380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:20.534380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:20.534380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:20.534380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:20.534380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:20.534380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:20.534380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:20.534380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:20.534380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:20.534380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:20.534380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:20.534380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:20.535521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:20.535521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:20.535521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:20.535521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:20.535521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:20.535521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:20.535521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:20.535521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:20.535521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:20.535521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:20.535521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:20.535521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:20.535521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:20.535521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:20.535521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:20.535521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:20.536462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:20.536462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:20.536462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:20.536462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:20.536462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:20.536462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:20.536462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:20.536462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:20.536462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:20.536462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:20.536462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:20.536462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:20.536462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 324/643] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:20.536462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317a9b1-360c-4c46-81c1-09341a707be7 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:20.549474 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73adf6e-3bd4-42fa-81d3-fba17c188160 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:20.551170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:20.583904 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b041c20-f8c5-4218-99b2-e8e7927b49e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:20.584589 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b041c20-f8c5-4218-99b2-e8e7927b49e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:20.584946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b041c20-f8c5-4218-99b2-e8e7927b49e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:20.585226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b041c20-f8c5-4218-99b2-e8e7927b49e3 None admin] PATH_INFO: `/v3/projects/449a5fac1acd478abea544446c08ecd9/users/d1047c7d18c1466d989cd895117cd807/roles/d1bb55f25dff4694b8675e0ef5fc4ec7` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:20.599260 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b041c20-f8c5-4218-99b2-e8e7927b49e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=449a5fac1acd478abea544446c08ecd9, user_id=d1047c7d18c1466d989cd895117cd807, role_id=d1bb55f25dff4694b8675e0ef5fc4ec7)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:20.600686 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b041c20-f8c5-4218-99b2-e8e7927b49e3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:20.634875 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317a9b1-360c-4c46-81c1-09341a707be7 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:20.668326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f6289-fc5c-4457-a07b-3939d2bf1eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:20.669469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f6289-fc5c-4457-a07b-3939d2bf1eef None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:20.669749 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f6289-fc5c-4457-a07b-3939d2bf1eef None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:20.670055 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f6289-fc5c-4457-a07b-3939d2bf1eef None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:20.670718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f6289-fc5c-4457-a07b-3939d2bf1eef None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:20.671024 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f6289-fc5c-4457-a07b-3939d2bf1eef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:20.672387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f6289-fc5c-4457-a07b-3939d2bf1eef None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:20.680518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 325/644] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:46:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:20.681386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f6289-fc5c-4457-a07b-3939d2bf1eef None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:20.705071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 320/645] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:46:20 2022] PUT /identity/v3/projects/449a5fac1acd478abea544446c08ecd9/users/d1047c7d18c1466d989cd895117cd807/roles/d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:46:20.709339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b041c20-f8c5-4218-99b2-e8e7927b49e3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:20.724380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:20.800988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f6289-fc5c-4457-a07b-3939d2bf1eef None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:20.829033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b041c20-f8c5-4218-99b2-e8e7927b49e3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:20.844883 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4c8b4-dba9-4c84-bc8b-d28050a68199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:20.845533 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4c8b4-dba9-4c84-bc8b-d28050a68199 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:20.845814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4c8b4-dba9-4c84-bc8b-d28050a68199 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:20.846129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4c8b4-dba9-4c84-bc8b-d28050a68199 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:20.854591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4c8b4-dba9-4c84-bc8b-d28050a68199 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:20.856060 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4c8b4-dba9-4c84-bc8b-d28050a68199 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:20.868797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-6fc4c8b4-dba9-4c84-bc8b-d28050a68199 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 14 11:46:20.868797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:20.868797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:20.868797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:20.868797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:20.868797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:20.868797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:20.868797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:20.868797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:20.868797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:20.868797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:20.868797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:20.868797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:20.868797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:20.868797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:20.868797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:20.870069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:20.870069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:20.870069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:20.870069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:20.870069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:20.870069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:20.870069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:20.870069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:20.870069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:20.870069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:20.870069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:20.870069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:20.870069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:20.870069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:20.870069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:20.870069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:20.870069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:20.870069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:20.871074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:20.871074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:20.871074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:20.871074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:20.871074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:20.871074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:20.871074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:20.871074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:20.871074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 14 11:46:20.871074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:20.873359 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12629482-7bbf-40dc-aeca-4dbf819abd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:20.874328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12629482-7bbf-40dc-aeca-4dbf819abd9b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:20.874328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12629482-7bbf-40dc-aeca-4dbf819abd9b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:20.874328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12629482-7bbf-40dc-aeca-4dbf819abd9b None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:20.875272 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12629482-7bbf-40dc-aeca-4dbf819abd9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:20.875366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 326/646] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:46:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:20.875563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12629482-7bbf-40dc-aeca-4dbf819abd9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:20.876934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4c8b4-dba9-4c84-bc8b-d28050a68199 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:20.877062 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12629482-7bbf-40dc-aeca-4dbf819abd9b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:20.892535 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 321/647] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:20.893201 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12629482-7bbf-40dc-aeca-4dbf819abd9b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:20.908644 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:20.977931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:20.987512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4c8b4-dba9-4c84-bc8b-d28050a68199 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:21.009654 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12629482-7bbf-40dc-aeca-4dbf819abd9b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:21.021725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9bc56-a1fe-4de3-be3d-afcb8efdedce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:21.022487 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9bc56-a1fe-4de3-be3d-afcb8efdedce None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:21.022876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9bc56-a1fe-4de3-be3d-afcb8efdedce None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:21.023270 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9bc56-a1fe-4de3-be3d-afcb8efdedce None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:21.024041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9bc56-a1fe-4de3-be3d-afcb8efdedce None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:21.024454 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9bc56-a1fe-4de3-be3d-afcb8efdedce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:21.026119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9bc56-a1fe-4de3-be3d-afcb8efdedce None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:21.036913 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 327/648] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:46:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:21.037981 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9bc56-a1fe-4de3-be3d-afcb8efdedce None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:21.052438 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9cbfd-638d-493e-b2c0-d9bf21f474b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:21.053074 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9cbfd-638d-493e-b2c0-d9bf21f474b4 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:21.053364 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9cbfd-638d-493e-b2c0-d9bf21f474b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:21.053643 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9cbfd-638d-493e-b2c0-d9bf21f474b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:21.064413 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9cbfd-638d-493e-b2c0-d9bf21f474b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:21.065852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9cbfd-638d-493e-b2c0-d9bf21f474b4 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:21.080109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-15d9cbfd-638d-493e-b2c0-d9bf21f474b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:21.080109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:21.080109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:21.080109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:21.080109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:21.080109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:21.080109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:21.080109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:21.080109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:21.080109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:21.080109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:21.080109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:21.080109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:21.080109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:21.080109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:21.080109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:21.081125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:21.081125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:21.081125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:21.081125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:21.081125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:21.081125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:21.081125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:21.081125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:21.081125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:21.081125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:21.081125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:21.081125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:21.081125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:21.081125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:21.081125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:21.081125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:21.081125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:21.082475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:21.082475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:21.082475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:21.085852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 322/649] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:21.086738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9cbfd-638d-493e-b2c0-d9bf21f474b4 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:21.146119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9bc56-a1fe-4de3-be3d-afcb8efdedce None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:21.152043 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:21.182053 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02932f-ce60-4c63-bba3-01486f8b6fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:21.182670 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02932f-ce60-4c63-bba3-01486f8b6fca None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:21.182955 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02932f-ce60-4c63-bba3-01486f8b6fca None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:21.183233 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02932f-ce60-4c63-bba3-01486f8b6fca None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:21.183974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02932f-ce60-4c63-bba3-01486f8b6fca None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:21.184288 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02932f-ce60-4c63-bba3-01486f8b6fca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:21.185790 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02932f-ce60-4c63-bba3-01486f8b6fca None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:21.197093 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9cbfd-638d-493e-b2c0-d9bf21f474b4 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:21.207720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 328/650] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Sep 14 11:46:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:21.207720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02932f-ce60-4c63-bba3-01486f8b6fca None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:21.221668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:21.237911 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa68475-6fd1-4ec6-9672-0fe91030445d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:21.238520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa68475-6fd1-4ec6-9672-0fe91030445d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:21.238922 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa68475-6fd1-4ec6-9672-0fe91030445d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:21.239210 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa68475-6fd1-4ec6-9672-0fe91030445d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:21.239934 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa68475-6fd1-4ec6-9672-0fe91030445d None admin] RBAC: Adding query filter params (role.id=d1bb55f25dff4694b8675e0ef5fc4ec7, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=38b093bc4c1d4ebcbd2a6bb16b70c029) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:21.240251 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa68475-6fd1-4ec6-9672-0fe91030445d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:21.241690 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa68475-6fd1-4ec6-9672-0fe91030445d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:21.258225 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 323/651] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:46:21 2022] GET /identity/v3/role_assignments?user.id=38b093bc4c1d4ebcbd2a6bb16b70c029&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:21.258225 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa68475-6fd1-4ec6-9672-0fe91030445d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:21.332095 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02932f-ce60-4c63-bba3-01486f8b6fca None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:21.372403 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcee7a8-bd7e-42f7-8694-136c5cbbe77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:21.373294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcee7a8-bd7e-42f7-8694-136c5cbbe77a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:21.373652 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcee7a8-bd7e-42f7-8694-136c5cbbe77a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:21.374046 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcee7a8-bd7e-42f7-8694-136c5cbbe77a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:21.386446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcee7a8-bd7e-42f7-8694-136c5cbbe77a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:21.388686 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcee7a8-bd7e-42f7-8694-136c5cbbe77a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:21.389930 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa68475-6fd1-4ec6-9672-0fe91030445d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:21.407326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-5fcee7a8-bd7e-42f7-8694-136c5cbbe77a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:21.407326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:21.407326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:21.407326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:21.407326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:21.407326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:21.407326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:21.407326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:21.407326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:21.407326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:21.407326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:21.407326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:21.407326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:21.407326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:21.407326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:21.407326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:21.408302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:21.408302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:21.408302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:21.408302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:21.408302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:21.408302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:21.408302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:21.408302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:21.408302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:21.408302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:21.408302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:21.408302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:21.408302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:21.408302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:21.408302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:21.408302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:21.408302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:21.409204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:21.409204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:21.409204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:21.409204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 329/652] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:21.410567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcee7a8-bd7e-42f7-8694-136c5cbbe77a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:21.422013 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:21.432067 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ede363-5359-412c-999a-3f449d13ed1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:21.432803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ede363-5359-412c-999a-3f449d13ed1f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:21.433091 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ede363-5359-412c-999a-3f449d13ed1f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:21.433366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ede363-5359-412c-999a-3f449d13ed1f None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:21.434157 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ede363-5359-412c-999a-3f449d13ed1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:21.435407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ede363-5359-412c-999a-3f449d13ed1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:21.436084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ede363-5359-412c-999a-3f449d13ed1f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:21.450864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 324/653] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:21.451728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ede363-5359-412c-999a-3f449d13ed1f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:21.496543 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:21.539511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcee7a8-bd7e-42f7-8694-136c5cbbe77a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:21.576414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ede363-5359-412c-999a-3f449d13ed1f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:21.577692 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92c1ef-80fe-470a-89e8-a356f3d47b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:21.578597 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92c1ef-80fe-470a-89e8-a356f3d47b84 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:21.578944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92c1ef-80fe-470a-89e8-a356f3d47b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:21.579295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92c1ef-80fe-470a-89e8-a356f3d47b84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:21.625124 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4191b-9842-49b1-b378-68d74241dab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:21.626303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4191b-9842-49b1-b378-68d74241dab4 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:21.626589 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4191b-9842-49b1-b378-68d74241dab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:21.626869 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4191b-9842-49b1-b378-68d74241dab4 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:21.627976 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4191b-9842-49b1-b378-68d74241dab4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:21.628292 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4191b-9842-49b1-b378-68d74241dab4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:21.630458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4191b-9842-49b1-b378-68d74241dab4 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:21.647185 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 325/654] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:21.648156 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4191b-9842-49b1-b378-68d74241dab4 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:21.661380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92c1ef-80fe-470a-89e8-a356f3d47b84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:21.662924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92c1ef-80fe-470a-89e8-a356f3d47b84 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:21.710950 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:21.752333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:21.767567 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4191b-9842-49b1-b378-68d74241dab4 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:21.805449 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac17ad0-27ff-429a-8f84-47649a56d828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:21.806174 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 330/655] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:46:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:46:21.806300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac17ad0-27ff-429a-8f84-47649a56d828 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:21.806546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac17ad0-27ff-429a-8f84-47649a56d828 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:21.809878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac17ad0-27ff-429a-8f84-47649a56d828 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:21.810006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92c1ef-80fe-470a-89e8-a356f3d47b84 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:21.817545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac17ad0-27ff-429a-8f84-47649a56d828 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:21.819966 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac17ad0-27ff-429a-8f84-47649a56d828 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:21.838441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-6ac17ad0-27ff-429a-8f84-47649a56d828 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 11:46:21.838441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:21.838441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:21.838441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:21.838441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:21.838441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:21.838441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:21.838441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:21.838441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:21.838441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:21.838441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:21.838441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:21.838441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:21.838441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:21.838441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:21.838441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:21.839685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:21.839685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:21.839685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:21.839685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:21.839685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:21.839685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:21.839685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:21.839685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:21.839685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:21.839685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:21.839685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:21.839685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:21.839685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:21.839685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:21.839685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:21.839685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:21.839685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:21.839685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:21.840659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:21.840659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:21.840659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:21.840659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:21.840659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:21.840659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:21.840659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:21.840659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:21.840659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 11:46:21.840659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:21.840659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 326/656] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:46:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:21.840659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac17ad0-27ff-429a-8f84-47649a56d828 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:21.915321 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92c1ef-80fe-470a-89e8-a356f3d47b84 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:21.957477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5aedd4-8fdb-45de-9026-3cc010833abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:21.958145 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5aedd4-8fdb-45de-9026-3cc010833abb None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:21.958429 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5aedd4-8fdb-45de-9026-3cc010833abb None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:21.958703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5aedd4-8fdb-45de-9026-3cc010833abb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:21.967004 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac17ad0-27ff-429a-8f84-47649a56d828 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:21.967175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5aedd4-8fdb-45de-9026-3cc010833abb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:21.968871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5aedd4-8fdb-45de-9026-3cc010833abb None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:21.977307 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:21.982795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-6d5aedd4-8fdb-45de-9026-3cc010833abb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 11:46:21.982795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:21.982795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:21.982795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:21.982795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:21.982795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:21.982795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:21.982795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:21.982795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:21.982795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:21.982795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:21.982795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:21.982795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:21.982795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:21.982795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:21.982795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:21.983742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:21.983742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:21.983742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:21.983742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:21.983742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:21.983742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:21.983742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:21.983742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:21.983742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:21.983742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:21.983742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:21.983742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:21.983742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:21.983742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:21.983742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:21.983742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:21.983742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:21.983742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:21.985069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:21.985069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:21.985069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:21.985069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:21.985069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:21.985069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:21.985069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:21.985069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:21.985069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 11:46:21.985069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:21.985069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 331/657] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:21.985069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5aedd4-8fdb-45de-9026-3cc010833abb None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:22.006464 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b038b2-c323-421e-ba16-054567b6bc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:22.007520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b038b2-c323-421e-ba16-054567b6bc62 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:22.008012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b038b2-c323-421e-ba16-054567b6bc62 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:22.008484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b038b2-c323-421e-ba16-054567b6bc62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:22.019649 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:22.023780 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b038b2-c323-421e-ba16-054567b6bc62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:22.025445 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b038b2-c323-421e-ba16-054567b6bc62 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:22.042706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-a7b038b2-c323-421e-ba16-054567b6bc62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:22.042706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:22.042706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:22.042706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:22.042706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:22.042706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:22.042706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:22.042706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:22.042706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:22.042706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:22.042706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:22.042706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:22.042706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:22.042706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:22.042706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:22.042706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:22.043683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:22.043683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:22.043683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:22.043683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:22.043683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:22.043683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:22.043683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:22.043683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:22.043683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:22.043683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:22.043683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:22.043683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:22.043683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:22.043683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:22.043683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:22.043683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:22.043683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:22.044615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:22.044615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:22.044615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:22.047132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 327/658] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:22.048102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b038b2-c323-421e-ba16-054567b6bc62 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:22.108303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5aedd4-8fdb-45de-9026-3cc010833abb None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:22.146014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca699c-0895-4540-8aff-162a1c3413aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:22.146624 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca699c-0895-4540-8aff-162a1c3413aa None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:22.147023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca699c-0895-4540-8aff-162a1c3413aa None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:22.147223 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca699c-0895-4540-8aff-162a1c3413aa None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:22.147958 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca699c-0895-4540-8aff-162a1c3413aa None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:22.148271 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca699c-0895-4540-8aff-162a1c3413aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:22.149916 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca699c-0895-4540-8aff-162a1c3413aa None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:22.158569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b038b2-c323-421e-ba16-054567b6bc62 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:22.170281 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 332/659] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Sep 14 11:46:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:22.170920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca699c-0895-4540-8aff-162a1c3413aa None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:22.198453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831fc162-1323-4598-818c-90d0ee5cb69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:22.199669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831fc162-1323-4598-818c-90d0ee5cb69d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:22.200204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831fc162-1323-4598-818c-90d0ee5cb69d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:22.200826 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831fc162-1323-4598-818c-90d0ee5cb69d None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:22.202243 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831fc162-1323-4598-818c-90d0ee5cb69d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:22.202870 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831fc162-1323-4598-818c-90d0ee5cb69d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:22.204944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831fc162-1323-4598-818c-90d0ee5cb69d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:22.226406 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 328/660] 10.222.0.62 () {56 vars in 1139 bytes} [Wed Sep 14 11:46:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:22.227611 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831fc162-1323-4598-818c-90d0ee5cb69d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:22.251071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:22.271016 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:22.292383 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca699c-0895-4540-8aff-162a1c3413aa None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:22.342172 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2e9ff-4035-491a-9035-73ee18755930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:22.343398 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2e9ff-4035-491a-9035-73ee18755930 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:22.344278 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2e9ff-4035-491a-9035-73ee18755930 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:22.345006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2e9ff-4035-491a-9035-73ee18755930 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:22.345851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2e9ff-4035-491a-9035-73ee18755930 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:22.346374 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2e9ff-4035-491a-9035-73ee18755930 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:22.346602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831fc162-1323-4598-818c-90d0ee5cb69d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:22.348537 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2e9ff-4035-491a-9035-73ee18755930 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:22.364665 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 333/661] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:22.365903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2e9ff-4035-491a-9035-73ee18755930 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:22.395646 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e8348-9bb7-416a-a5d0-c5c9e5503902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:22.396923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e8348-9bb7-416a-a5d0-c5c9e5503902 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:22.397733 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e8348-9bb7-416a-a5d0-c5c9e5503902 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:22.398661 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e8348-9bb7-416a-a5d0-c5c9e5503902 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:22.483780 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e8348-9bb7-416a-a5d0-c5c9e5503902 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:22.485142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e8348-9bb7-416a-a5d0-c5c9e5503902 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:22.488696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2e9ff-4035-491a-9035-73ee18755930 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:22.504191 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:22.535154 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2cd08-ffca-4a98-84ed-af96892a3f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:22.536624 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2cd08-ffca-4a98-84ed-af96892a3f11 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:22.537053 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2cd08-ffca-4a98-84ed-af96892a3f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:22.537713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2cd08-ffca-4a98-84ed-af96892a3f11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:22.559706 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:22.594250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 329/662] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:46:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:46:22.595146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e8348-9bb7-416a-a5d0-c5c9e5503902 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:22.633601 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2cd08-ffca-4a98-84ed-af96892a3f11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:22.635344 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2cd08-ffca-4a98-84ed-af96892a3f11 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:22.706502 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e8348-9bb7-416a-a5d0-c5c9e5503902 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:22.726244 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:22.745406 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 334/663] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:46:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:46:22.745406 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2cd08-ffca-4a98-84ed-af96892a3f11 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:22.750123 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53e18a-746c-4c01-a7d2-1cd3b6964888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:22.750829 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53e18a-746c-4c01-a7d2-1cd3b6964888 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:22.751119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53e18a-746c-4c01-a7d2-1cd3b6964888 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:22.751390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53e18a-746c-4c01-a7d2-1cd3b6964888 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:22.759939 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53e18a-746c-4c01-a7d2-1cd3b6964888 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:22.762521 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53e18a-746c-4c01-a7d2-1cd3b6964888 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:22.775090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-1b53e18a-746c-4c01-a7d2-1cd3b6964888 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:22.775090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:22.775090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:22.775090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:22.775090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:22.775090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:22.775090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:22.775090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:22.775090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:22.775090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:22.775090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:22.775090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:22.775090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:22.775090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:22.775090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:22.776018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:22.776018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:22.776018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:22.776018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:22.776018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:22.776018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:22.776018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:22.776018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:22.776018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:22.776018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:22.776018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:22.776018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:22.776018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:22.776018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:22.776018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:22.776018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:22.776018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:22.776917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:22.776917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:22.776917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:22.776917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:22.776917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:22.776917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:22.776917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:22.776917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:22.781090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 330/664] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:22.782192 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53e18a-746c-4c01-a7d2-1cd3b6964888 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:22.797952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:22.864084 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2cd08-ffca-4a98-84ed-af96892a3f11 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:22.899058 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77296449-c47e-4b29-978c-ea1adebeead0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:22.900064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77296449-c47e-4b29-978c-ea1adebeead0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:22.900547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77296449-c47e-4b29-978c-ea1adebeead0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:22.901030 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77296449-c47e-4b29-978c-ea1adebeead0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:22.901488 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53e18a-746c-4c01-a7d2-1cd3b6964888 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:22.912306 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77296449-c47e-4b29-978c-ea1adebeead0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:22.914050 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77296449-c47e-4b29-978c-ea1adebeead0 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:22.929000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-77296449-c47e-4b29-978c-ea1adebeead0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:22.929000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:22.929000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:22.929000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:22.929000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:22.929000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:22.929000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:22.929000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:22.929000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:22.929000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:22.929000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:22.929000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:22.929000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:22.929000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:22.929000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:22.929000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:22.935638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:22.935638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:22.935638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:22.935638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:22.935638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:22.935638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:22.935638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:22.935638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:22.935638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:22.935638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:22.935638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:22.935638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:22.935638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:22.935638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:22.935638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:22.935638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:22.935638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:22.936729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:22.936729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:22.936729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:22.936729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 335/665] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:22.936729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77296449-c47e-4b29-978c-ea1adebeead0 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:22.942766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de0d37-8ac0-4205-a23e-81fe9b6aea81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:22.943407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de0d37-8ac0-4205-a23e-81fe9b6aea81 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:22.943692 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de0d37-8ac0-4205-a23e-81fe9b6aea81 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:22.943971 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de0d37-8ac0-4205-a23e-81fe9b6aea81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:22.956020 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80de0d37-8ac0-4205-a23e-81fe9b6aea81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:22.957462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80de0d37-8ac0-4205-a23e-81fe9b6aea81 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:22.973745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-80de0d37-8ac0-4205-a23e-81fe9b6aea81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:22.973745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:22.973745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:22.973745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:22.973745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:22.973745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:22.973745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:22.973745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:22.973745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:22.973745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:22.973745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:22.973745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:22.973745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:22.973745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:22.973745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:22.973745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:22.974711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:22.974711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:22.974711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:22.974711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:22.974711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:22.974711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:22.974711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:22.974711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:22.974711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:22.974711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:22.974711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:22.974711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:22.974711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:22.974711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:22.974711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:22.974711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:22.974711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:22.975605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:22.975605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:22.975605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:22.976404 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 331/666] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:22.977881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0b34c-aa99-48bd-ab15-c554596c614c None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:22.978261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0b34c-aa99-48bd-ab15-c554596c614c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:22.978604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0b34c-aa99-48bd-ab15-c554596c614c None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:22.981823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 332/667] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:22.981823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0b34c-aa99-48bd-ab15-c554596c614c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:23.013317 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:23.039160 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:23.044768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77296449-c47e-4b29-978c-ea1adebeead0 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:23.079593 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f0f29-6d9e-4d10-ac9d-02f8e29cdb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:23.080321 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f0f29-6d9e-4d10-ac9d-02f8e29cdb84 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:23.080663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f0f29-6d9e-4d10-ac9d-02f8e29cdb84 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:23.081005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f0f29-6d9e-4d10-ac9d-02f8e29cdb84 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:23.081764 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f0f29-6d9e-4d10-ac9d-02f8e29cdb84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:23.082168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f0f29-6d9e-4d10-ac9d-02f8e29cdb84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:23.083618 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f0f29-6d9e-4d10-ac9d-02f8e29cdb84 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:23.093887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0b34c-aa99-48bd-ab15-c554596c614c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:23.101465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 336/668] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:23.101465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f0f29-6d9e-4d10-ac9d-02f8e29cdb84 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:23.136803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a807a-c843-45bb-8ae6-415fadb9fe05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:23.138209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a807a-c843-45bb-8ae6-415fadb9fe05 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:23.138657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a807a-c843-45bb-8ae6-415fadb9fe05 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:23.139079 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a807a-c843-45bb-8ae6-415fadb9fe05 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:23.139882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a807a-c843-45bb-8ae6-415fadb9fe05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:23.140335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a807a-c843-45bb-8ae6-415fadb9fe05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:23.141913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a807a-c843-45bb-8ae6-415fadb9fe05 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:23.159365 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 333/669] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:23.161053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9216b-60a6-49f0-a0b5-9efbe70283f0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:23.161515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9216b-60a6-49f0-a0b5-9efbe70283f0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:23.162009 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9216b-60a6-49f0-a0b5-9efbe70283f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:23.180543 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-15c9216b-60a6-49f0-a0b5-9efbe70283f0 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:23.181743 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-15c9216b-60a6-49f0-a0b5-9efbe70283f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:23.206712 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f0f29-6d9e-4d10-ac9d-02f8e29cdb84 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:23.246457 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5ccab-32b7-4d12-836d-6f7253effb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:23.247602 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5ccab-32b7-4d12-836d-6f7253effb7c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:23.247888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5ccab-32b7-4d12-836d-6f7253effb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:23.248158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5ccab-32b7-4d12-836d-6f7253effb7c None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:23.248804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5ccab-32b7-4d12-836d-6f7253effb7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:23.249112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5ccab-32b7-4d12-836d-6f7253effb7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:23.250488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5ccab-32b7-4d12-836d-6f7253effb7c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:23.253939 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 334/670] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:23.254975 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9216b-60a6-49f0-a0b5-9efbe70283f0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:23.274935 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 337/671] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:23.274935 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5ccab-32b7-4d12-836d-6f7253effb7c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:23.287504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:23.294096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:23.371295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9216b-60a6-49f0-a0b5-9efbe70283f0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:23.407647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7da428-71ae-4531-8463-1864f8881d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:23.408563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7da428-71ae-4531-8463-1864f8881d2f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:23.408696 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7da428-71ae-4531-8463-1864f8881d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:23.409001 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7da428-71ae-4531-8463-1864f8881d2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:23.409726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7da428-71ae-4531-8463-1864f8881d2f None admin] RBAC: Adding query filter params (role.id=d1bb55f25dff4694b8675e0ef5fc4ec7, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=4b05d0e65e9645f3810a6d96c8fed9fc) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:23.410072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7da428-71ae-4531-8463-1864f8881d2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:23.411471 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7da428-71ae-4531-8463-1864f8881d2f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:23.412284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5ccab-32b7-4d12-836d-6f7253effb7c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:23.422011 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 335/672] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:46:23 2022] GET /identity/v3/role_assignments?user.id=4b05d0e65e9645f3810a6d96c8fed9fc&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:23.423132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7da428-71ae-4531-8463-1864f8881d2f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:23.488113 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a29a9f-a37f-4091-952c-30a1b6caf504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:23.490375 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a29a9f-a37f-4091-952c-30a1b6caf504 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:23.491458 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a29a9f-a37f-4091-952c-30a1b6caf504 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:23.491925 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a29a9f-a37f-4091-952c-30a1b6caf504 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:23.507740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a29a9f-a37f-4091-952c-30a1b6caf504 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:23.509797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a29a9f-a37f-4091-952c-30a1b6caf504 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:23.526620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-23a29a9f-a37f-4091-952c-30a1b6caf504 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:23.526620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:23.526620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:23.526620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:23.526620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:23.526620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:23.526620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:23.526620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:23.526620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:23.526620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:23.526620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:23.526620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:23.526620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:23.526620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:23.526620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:23.527587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:23.527587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:23.527587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:23.527587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:23.527587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:23.527587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:23.527587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:23.527587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:23.527587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:23.527587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:23.527587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:23.527587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:23.527587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:23.527587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:23.527587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:23.527587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:23.527587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:23.528621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:23.528621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:23.528621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:23.528621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:23.528621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:23.528621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:23.528621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:23.528621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:23.535787 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 338/673] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:23.539006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a29a9f-a37f-4091-952c-30a1b6caf504 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:23.618816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:23.635063 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7da428-71ae-4531-8463-1864f8881d2f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:23.645998 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:23.673138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a29a9f-a37f-4091-952c-30a1b6caf504 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:23.678049 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a16674-65c3-4416-9cfc-17718fc324d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:23.678716 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a16674-65c3-4416-9cfc-17718fc324d9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:23.679000 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a16674-65c3-4416-9cfc-17718fc324d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:23.679277 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a16674-65c3-4416-9cfc-17718fc324d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:23.687288 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a16674-65c3-4416-9cfc-17718fc324d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:23.689478 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a16674-65c3-4416-9cfc-17718fc324d9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:23.701029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-22a16674-65c3-4416-9cfc-17718fc324d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:23.701029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:23.701029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:23.701029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:23.701029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:23.701029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:23.701029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:23.701029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:23.701029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:23.701029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:23.701029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:23.701029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:23.701029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:23.701029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:23.701029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:23.701947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:23.701947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:23.701947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:23.701947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:23.701947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:23.701947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:23.701947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:23.701947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:23.701947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:23.701947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:23.701947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:23.701947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:23.701947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:23.701947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:23.701947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:23.701947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:23.701947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:23.702853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:23.702853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:23.702853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:23.702853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:23.702853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:23.702853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:23.702853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:23.702853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:23.702853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 336/674] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:23.703798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a16674-65c3-4416-9cfc-17718fc324d9 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:23.711693 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0bdb4-617d-4a55-9004-b97e7e4ecaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:23.712313 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0bdb4-617d-4a55-9004-b97e7e4ecaf5 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:23.712601 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0bdb4-617d-4a55-9004-b97e7e4ecaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:23.712887 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0bdb4-617d-4a55-9004-b97e7e4ecaf5 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:23.713589 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0bdb4-617d-4a55-9004-b97e7e4ecaf5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:23.713933 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0bdb4-617d-4a55-9004-b97e7e4ecaf5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:23.715392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0bdb4-617d-4a55-9004-b97e7e4ecaf5 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:23.728252 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 339/675] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:46:23.729496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f176c-9d82-4036-9690-3251028524e6 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:23.729874 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f176c-9d82-4036-9690-3251028524e6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:23.730296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f176c-9d82-4036-9690-3251028524e6 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:23.732292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 340/676] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:23.733016 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f176c-9d82-4036-9690-3251028524e6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:23.810736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a16674-65c3-4416-9cfc-17718fc324d9 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:23.850395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f176c-9d82-4036-9690-3251028524e6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:23.853089 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295d543-ddb7-4642-86fa-1292a96276da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:23.853732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295d543-ddb7-4642-86fa-1292a96276da None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:23.854059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295d543-ddb7-4642-86fa-1292a96276da None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:23.854336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295d543-ddb7-4642-86fa-1292a96276da None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:23.865505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1295d543-ddb7-4642-86fa-1292a96276da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:23.870928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-1295d543-ddb7-4642-86fa-1292a96276da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:23.870928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:23.870928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:23.870928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:23.870928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:23.870928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:23.870928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:23.870928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:23.870928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:23.870928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:23.870928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:23.870928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:23.870928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:23.870928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:23.870928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:23.870928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:23.872448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:23.872448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:23.872448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:23.872448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:23.872448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:23.872448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:23.872448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:23.872448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:23.872448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:23.872448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:23.872448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:23.872448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:23.872448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:23.872448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:23.872448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:23.872448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:23.873299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:23.873299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:23.873299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:23.873299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:23.873299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:23.873299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:23.873299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:23.873299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:23.873299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:23.873299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:23.873299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:23.873299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:23.873299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 337/677] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:23.874038 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295d543-ddb7-4642-86fa-1292a96276da None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:23.877667 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:23.893155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9c187-ee9b-4aa3-915a-f0897bf412f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:23.893912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9c187-ee9b-4aa3-915a-f0897bf412f8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:23.894175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9c187-ee9b-4aa3-915a-f0897bf412f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:23.894460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9c187-ee9b-4aa3-915a-f0897bf412f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:23.895153 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9c187-ee9b-4aa3-915a-f0897bf412f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:23.895463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9c187-ee9b-4aa3-915a-f0897bf412f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:23.896898 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9c187-ee9b-4aa3-915a-f0897bf412f8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:23.899560 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:23.914131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 341/678] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:23.914131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51f059-5020-49e2-b588-f79cc25dd29c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:23.914131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51f059-5020-49e2-b588-f79cc25dd29c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:23.914131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51f059-5020-49e2-b588-f79cc25dd29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:23.929644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-bd51f059-5020-49e2-b588-f79cc25dd29c None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:23.930958 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-bd51f059-5020-49e2-b588-f79cc25dd29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:23.986244 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295d543-ddb7-4642-86fa-1292a96276da None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:24.004120 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 342/679] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:24.007021 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51f059-5020-49e2-b588-f79cc25dd29c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:24.029975 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3736cd9-e34a-4aab-b17e-d4d754df9074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:24.031330 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3736cd9-e34a-4aab-b17e-d4d754df9074 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:24.031824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3736cd9-e34a-4aab-b17e-d4d754df9074 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:24.032279 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3736cd9-e34a-4aab-b17e-d4d754df9074 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:24.033207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3736cd9-e34a-4aab-b17e-d4d754df9074 None admin] RBAC: Adding query filter params (role.id=d1bb55f25dff4694b8675e0ef5fc4ec7, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=d27c7ff726e443e89da94f282b5cddab) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:24.034257 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3736cd9-e34a-4aab-b17e-d4d754df9074 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:24.036173 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3736cd9-e34a-4aab-b17e-d4d754df9074 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:24.054147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 338/680] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:46:23 2022] GET /identity/v3/role_assignments?user.id=d27c7ff726e443e89da94f282b5cddab&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:24.054147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3736cd9-e34a-4aab-b17e-d4d754df9074 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:24.202611 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:24.204811 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51f059-5020-49e2-b588-f79cc25dd29c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:24.247052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3736cd9-e34a-4aab-b17e-d4d754df9074 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:24.251148 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:24.261279 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b7407-69e5-4ebf-90d8-4a0f3ffb2500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:24.262802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b7407-69e5-4ebf-90d8-4a0f3ffb2500 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:24.279005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b7407-69e5-4ebf-90d8-4a0f3ffb2500 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:24.279005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b7407-69e5-4ebf-90d8-4a0f3ffb2500 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:24.279005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b7407-69e5-4ebf-90d8-4a0f3ffb2500 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:24.279005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b7407-69e5-4ebf-90d8-4a0f3ffb2500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:24.279005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b7407-69e5-4ebf-90d8-4a0f3ffb2500 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:24.279843 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 343/681] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:46:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:24.285308 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b7407-69e5-4ebf-90d8-4a0f3ffb2500 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:24.290349 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ddd3b-45ac-4c0b-89fa-a2c87ec07637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:24.290608 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ddd3b-45ac-4c0b-89fa-a2c87ec07637 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:24.290892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ddd3b-45ac-4c0b-89fa-a2c87ec07637 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:24.291165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ddd3b-45ac-4c0b-89fa-a2c87ec07637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:24.291892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ddd3b-45ac-4c0b-89fa-a2c87ec07637 None admin] RBAC: Adding query filter params (role.id=d1bb55f25dff4694b8675e0ef5fc4ec7, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=13f1840d92d94c3ebd16000a4889c377) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:24.292202 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ddd3b-45ac-4c0b-89fa-a2c87ec07637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:24.293647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ddd3b-45ac-4c0b-89fa-a2c87ec07637 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:24.301140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 339/682] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:46:24 2022] GET /identity/v3/role_assignments?user.id=13f1840d92d94c3ebd16000a4889c377&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:24.303430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ddd3b-45ac-4c0b-89fa-a2c87ec07637 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:24.422621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b7407-69e5-4ebf-90d8-4a0f3ffb2500 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:24.429633 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ddd3b-45ac-4c0b-89fa-a2c87ec07637 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:24.469108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:24.470788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d547ed-f49c-4ae0-baa1-4e17a28fc1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:24.471752 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d547ed-f49c-4ae0-baa1-4e17a28fc1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:24.472207 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d547ed-f49c-4ae0-baa1-4e17a28fc1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:24.472514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d547ed-f49c-4ae0-baa1-4e17a28fc1d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:24.476189 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47b7dc-0528-4f0a-8250-05e0c4ccf5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:24.489534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47b7dc-0528-4f0a-8250-05e0c4ccf5cf None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:24.489534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47b7dc-0528-4f0a-8250-05e0c4ccf5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:24.489534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47b7dc-0528-4f0a-8250-05e0c4ccf5cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:24.489782 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d547ed-f49c-4ae0-baa1-4e17a28fc1d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:24.489782 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d547ed-f49c-4ae0-baa1-4e17a28fc1d1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:24.490809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47b7dc-0528-4f0a-8250-05e0c4ccf5cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:24.495897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-fc47b7dc-0528-4f0a-8250-05e0c4ccf5cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:46:24.495897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:24.495897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:24.495897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:24.495897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:24.495897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:24.495897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:24.495897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:24.495897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:24.495897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:24.495897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:24.495897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:24.495897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:24.495897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:24.495897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:24.495897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:24.497010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:24.497010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:24.497010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:24.497010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:24.497010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:24.497010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:24.497010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:24.497010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:24.497010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:24.497010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:24.497010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:24.497010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:24.497010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:24.497010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:24.497010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:24.497010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:24.497889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:24.497889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:24.497889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:24.497889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:24.497889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:24.497889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:24.497889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:24.497889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:24.497889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:24.497889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:24.497889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:46:24.497889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:24.497889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 340/683] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:46:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:24.504023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-25d547ed-f49c-4ae0-baa1-4e17a28fc1d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:24.504023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:24.504023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:24.504023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:24.504023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:24.504023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:24.504023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:24.504023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:24.504023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:24.504023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:24.504023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:24.504023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:24.504023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:24.504023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:24.504023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:24.504023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:24.504990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:24.504990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:24.504990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:24.504990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:24.504990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:24.504990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:24.504990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:24.504990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:24.504990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:24.504990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:24.504990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:24.504990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:24.504990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:24.504990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:24.504990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:24.504990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:24.504990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:24.505932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:24.505932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:24.505932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:24.505932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 344/684] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:24.511251 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47b7dc-0528-4f0a-8250-05e0c4ccf5cf None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:24.511612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d547ed-f49c-4ae0-baa1-4e17a28fc1d1 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:24.559807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:24.622756 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d547ed-f49c-4ae0-baa1-4e17a28fc1d1 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:24.633408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47b7dc-0528-4f0a-8250-05e0c4ccf5cf None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:24.664909 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55414284-1074-43af-a1a3-55fdae3ba4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:24.665528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55414284-1074-43af-a1a3-55fdae3ba4db None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:24.665817 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55414284-1074-43af-a1a3-55fdae3ba4db None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:24.666572 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55414284-1074-43af-a1a3-55fdae3ba4db None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:24.667132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55414284-1074-43af-a1a3-55fdae3ba4db None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:24.667490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55414284-1074-43af-a1a3-55fdae3ba4db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:24.669375 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55414284-1074-43af-a1a3-55fdae3ba4db None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:24.677110 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29797ae-c049-4505-8967-e6c93f128b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:24.678373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29797ae-c049-4505-8967-e6c93f128b5e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:24.678843 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29797ae-c049-4505-8967-e6c93f128b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:24.679298 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29797ae-c049-4505-8967-e6c93f128b5e None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:24.679990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 345/685] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:46:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:46:24.680325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29797ae-c049-4505-8967-e6c93f128b5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:24.680639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29797ae-c049-4505-8967-e6c93f128b5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:24.686112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29797ae-c049-4505-8967-e6c93f128b5e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:24.688960 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55414284-1074-43af-a1a3-55fdae3ba4db None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:24.706311 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 341/686] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:24.711773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29797ae-c049-4505-8967-e6c93f128b5e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:24.825969 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:24.847649 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55414284-1074-43af-a1a3-55fdae3ba4db None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:24.849323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:24.888776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca0f6e-ea3a-4c45-bf96-bf28c0259f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:24.889420 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca0f6e-ea3a-4c45-bf96-bf28c0259f4a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:24.889713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca0f6e-ea3a-4c45-bf96-bf28c0259f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:24.890083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca0f6e-ea3a-4c45-bf96-bf28c0259f4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:24.905064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca0f6e-ea3a-4c45-bf96-bf28c0259f4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:24.906567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca0f6e-ea3a-4c45-bf96-bf28c0259f4a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:24.924663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-bfca0f6e-ea3a-4c45-bf96-bf28c0259f4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:24.924663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:24.924663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:24.924663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:24.924663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:24.924663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:24.924663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:24.924663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:24.924663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:24.924663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:24.924663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:24.924663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:24.924663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:24.924663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:24.924663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:24.924663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:24.925644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:24.925644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:24.925644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:24.925644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:24.925644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:24.925644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:24.925644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:24.925644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:24.925644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:24.925644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:24.925644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:24.925644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:24.925644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:24.925644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:24.925644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:24.925644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:24.925644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:24.926594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:24.926594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:24.926594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:24.930589 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 346/687] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:24.932066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca0f6e-ea3a-4c45-bf96-bf28c0259f4a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:24.937907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29797ae-c049-4505-8967-e6c93f128b5e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:25.011776 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450f2ea-a69f-417a-83d5-d197ccd0ecc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:25.012803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450f2ea-a69f-417a-83d5-d197ccd0ecc9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:25.014114 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450f2ea-a69f-417a-83d5-d197ccd0ecc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:25.015147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450f2ea-a69f-417a-83d5-d197ccd0ecc9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:25.029025 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450f2ea-a69f-417a-83d5-d197ccd0ecc9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:25.030878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450f2ea-a69f-417a-83d5-d197ccd0ecc9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:25.043325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-0450f2ea-a69f-417a-83d5-d197ccd0ecc9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 11:46:25.043325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:25.043325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:25.043325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:25.043325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:25.043325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:25.043325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:25.043325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:25.043325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:25.043325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:25.043325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:25.043325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:25.043325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:25.043325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:25.043325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:25.043325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:25.044878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:25.044878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:25.044878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:25.044878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:25.044878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:25.044878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:25.044878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:25.044878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:25.044878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:25.044878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:25.044878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:25.044878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:25.044878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:25.044878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:25.044878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:25.044878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:25.044878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:25.044878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:25.045807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:25.045807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:25.045807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:25.045807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:25.045807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:25.045807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:25.045807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:25.045807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:25.045807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 11:46:25.045807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:25.048425 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 342/688] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:46:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:25.054003 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450f2ea-a69f-417a-83d5-d197ccd0ecc9 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:25.108023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca0f6e-ea3a-4c45-bf96-bf28c0259f4a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:25.164157 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81617b-a2f8-43a2-b73e-46eedc3d8473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:25.164827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81617b-a2f8-43a2-b73e-46eedc3d8473 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:25.165122 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81617b-a2f8-43a2-b73e-46eedc3d8473 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:25.165406 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81617b-a2f8-43a2-b73e-46eedc3d8473 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:25.166092 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81617b-a2f8-43a2-b73e-46eedc3d8473 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:25.166404 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81617b-a2f8-43a2-b73e-46eedc3d8473 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:25.167971 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81617b-a2f8-43a2-b73e-46eedc3d8473 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:25.181056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 347/689] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:46:25.186816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81617b-a2f8-43a2-b73e-46eedc3d8473 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:25.206941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:25.219462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450f2ea-a69f-417a-83d5-d197ccd0ecc9 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:25.245805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:25.288445 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc12fb-60c7-49a7-804c-48b2b4dd5b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:25.289067 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc12fb-60c7-49a7-804c-48b2b4dd5b7c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:25.289346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc12fb-60c7-49a7-804c-48b2b4dd5b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:25.289613 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc12fb-60c7-49a7-804c-48b2b4dd5b7c None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:25.290394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc12fb-60c7-49a7-804c-48b2b4dd5b7c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:25.290699 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc12fb-60c7-49a7-804c-48b2b4dd5b7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:25.292129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc12fb-60c7-49a7-804c-48b2b4dd5b7c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:25.323272 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81617b-a2f8-43a2-b73e-46eedc3d8473 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:25.341135 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 343/690] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Sep 14 11:46:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:25.346730 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc12fb-60c7-49a7-804c-48b2b4dd5b7c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:25.360413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e4ab5-6280-4d16-bdbd-66989b98101f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:25.361035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e4ab5-6280-4d16-bdbd-66989b98101f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:25.361317 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e4ab5-6280-4d16-bdbd-66989b98101f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:25.361635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e4ab5-6280-4d16-bdbd-66989b98101f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:25.369568 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e4ab5-6280-4d16-bdbd-66989b98101f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:25.371053 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e4ab5-6280-4d16-bdbd-66989b98101f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:25.384551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-1e3e4ab5-6280-4d16-bdbd-66989b98101f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 11:46:25.384551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:25.384551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:25.384551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:25.384551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:25.384551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:25.384551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:25.384551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:25.384551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:25.384551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:25.384551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:25.384551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:25.384551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:25.384551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:25.384551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:25.384551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:25.385555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:25.385555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:25.385555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:25.385555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:25.385555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:25.385555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:25.385555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:25.385555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:25.385555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:25.385555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:25.385555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:25.385555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:25.385555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:25.385555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:25.385555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:25.385555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:25.385555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:25.385555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:25.386617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:25.386617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:25.386617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:25.386617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:25.386617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:25.386617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:25.386617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:25.386617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:25.386617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 11:46:25.386617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:25.386617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 348/691] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Sep 14 11:46:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:25.395192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e4ab5-6280-4d16-bdbd-66989b98101f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:25.527055 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:25.556848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e4ab5-6280-4d16-bdbd-66989b98101f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:25.596512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7763bc-cf6e-497e-8fe4-496aef52e59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:25.597161 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc12fb-60c7-49a7-804c-48b2b4dd5b7c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:25.597958 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7763bc-cf6e-497e-8fe4-496aef52e59e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:25.598691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7763bc-cf6e-497e-8fe4-496aef52e59e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:25.599117 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7763bc-cf6e-497e-8fe4-496aef52e59e None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:25.600024 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7763bc-cf6e-497e-8fe4-496aef52e59e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:25.600557 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7763bc-cf6e-497e-8fe4-496aef52e59e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:25.602167 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7763bc-cf6e-497e-8fe4-496aef52e59e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:25.623972 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 349/692] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Sep 14 11:46:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:25.629447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7763bc-cf6e-497e-8fe4-496aef52e59e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:25.665724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02aebd-0e56-47c1-854d-dd0ff04e768e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:25.666744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02aebd-0e56-47c1-854d-dd0ff04e768e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:25.668152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02aebd-0e56-47c1-854d-dd0ff04e768e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:25.669203 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02aebd-0e56-47c1-854d-dd0ff04e768e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:25.746304 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:25.795787 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7763bc-cf6e-497e-8fe4-496aef52e59e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:25.844900 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a02aebd-0e56-47c1-854d-dd0ff04e768e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:25.845803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:25.846411 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a02aebd-0e56-47c1-854d-dd0ff04e768e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:25.855613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85f954-f73d-459f-bdad-7d7b8b6e7b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:25.856761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85f954-f73d-459f-bdad-7d7b8b6e7b95 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:25.857045 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85f954-f73d-459f-bdad-7d7b8b6e7b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:25.857322 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85f954-f73d-459f-bdad-7d7b8b6e7b95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:25.959487 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a85f954-f73d-459f-bdad-7d7b8b6e7b95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:25.961308 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a85f954-f73d-459f-bdad-7d7b8b6e7b95 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:25.988560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 344/693] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:46:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:46:25.989909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3d8f7-3f95-47f2-a442-8674cf33e912 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:25.990201 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3d8f7-3f95-47f2-a442-8674cf33e912 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:25.990523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3d8f7-3f95-47f2-a442-8674cf33e912 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:25.992011 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 345/694] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:25.993587 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3d8f7-3f95-47f2-a442-8674cf33e912 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:26.084619 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 350/695] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:46:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 11:46:26.086124 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c96c0-a6ea-4aa6-b6cc-0149882d1dfd None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:26.086525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c96c0-a6ea-4aa6-b6cc-0149882d1dfd None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:26.086875 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c96c0-a6ea-4aa6-b6cc-0149882d1dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:26.098322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:26.107499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f80c96c0-a6ea-4aa6-b6cc-0149882d1dfd None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:26.108688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f80c96c0-a6ea-4aa6-b6cc-0149882d1dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:26.135284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:26.141460 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3d8f7-3f95-47f2-a442-8674cf33e912 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:26.184500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16f0fa-1b31-4c03-b20d-a76ea718d681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:26.185319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16f0fa-1b31-4c03-b20d-a76ea718d681 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:26.185769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16f0fa-1b31-4c03-b20d-a76ea718d681 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:26.186158 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16f0fa-1b31-4c03-b20d-a76ea718d681 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:26.187654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 351/696] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:26.188803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c96c0-a6ea-4aa6-b6cc-0149882d1dfd None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:26.210280 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16f0fa-1b31-4c03-b20d-a76ea718d681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:26.211760 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16f0fa-1b31-4c03-b20d-a76ea718d681 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:26.243241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-de16f0fa-1b31-4c03-b20d-a76ea718d681 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:26.243241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:26.243241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:26.243241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:26.243241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:26.243241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:26.243241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:26.243241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:26.243241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:26.243241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:26.243241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:26.243241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:26.243241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:26.243241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:26.243241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:26.243241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:26.244253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:26.244253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:26.244253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:26.244253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:26.244253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:26.244253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:26.244253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:26.244253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:26.244253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:26.244253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:26.244253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:26.244253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:26.244253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:26.244253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:26.244253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:26.244253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:26.244253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:26.245146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:26.245146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:26.245146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:26.247625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 346/697] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:26.266866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16f0fa-1b31-4c03-b20d-a76ea718d681 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:26.327922 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c96c0-a6ea-4aa6-b6cc-0149882d1dfd None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:26.384045 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16f0fa-1b31-4c03-b20d-a76ea718d681 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:26.410587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c3bb8-ba6c-4957-97bc-ba679a0f39bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:26.411236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c3bb8-ba6c-4957-97bc-ba679a0f39bf None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:26.411526 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c3bb8-ba6c-4957-97bc-ba679a0f39bf None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:26.411806 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c3bb8-ba6c-4957-97bc-ba679a0f39bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:26.421082 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800f4de-a067-482d-8330-bc7e612150f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:26.421693 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800f4de-a067-482d-8330-bc7e612150f7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:26.422075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800f4de-a067-482d-8330-bc7e612150f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:26.422353 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800f4de-a067-482d-8330-bc7e612150f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:26.422998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800f4de-a067-482d-8330-bc7e612150f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:26.423305 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800f4de-a067-482d-8330-bc7e612150f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:26.424930 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800f4de-a067-482d-8330-bc7e612150f7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:26.429263 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c3bb8-ba6c-4957-97bc-ba679a0f39bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:26.432304 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:26.438888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c3bb8-ba6c-4957-97bc-ba679a0f39bf None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:26.441557 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 347/698] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:26.442789 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800f4de-a067-482d-8330-bc7e612150f7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:26.467738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-5d9c3bb8-ba6c-4957-97bc-ba679a0f39bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:26.467738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:26.467738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:26.467738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:26.467738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:26.467738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:26.467738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:26.467738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:26.467738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:26.467738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:26.467738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:26.467738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:26.467738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:26.467738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:26.467738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:26.467738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:26.470063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:26.470063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:26.470063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:26.470063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:26.470063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:26.470063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:26.470063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:26.470063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:26.470063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:26.470063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:26.470063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:26.470063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:26.470063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:26.470063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:26.470063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:26.470063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:26.470063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:26.472296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:26.472296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:26.472296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:26.472296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 352/699] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:26.479613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c3bb8-ba6c-4957-97bc-ba679a0f39bf None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:26.526232 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:26.548209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800f4de-a067-482d-8330-bc7e612150f7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:26.617665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93600e-5e92-48d7-a8bb-4d8f4a52b02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:26.618354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93600e-5e92-48d7-a8bb-4d8f4a52b02a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:26.618632 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93600e-5e92-48d7-a8bb-4d8f4a52b02a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:26.618904 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93600e-5e92-48d7-a8bb-4d8f4a52b02a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:26.628669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93600e-5e92-48d7-a8bb-4d8f4a52b02a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:26.631730 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93600e-5e92-48d7-a8bb-4d8f4a52b02a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:26.645394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-8f93600e-5e92-48d7-a8bb-4d8f4a52b02a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:26.645394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:26.645394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:26.645394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:26.645394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:26.645394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:26.645394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:26.645394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:26.645394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:26.645394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:26.645394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:26.645394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:26.645394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:26.645394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:26.645394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:26.646408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:26.646408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:26.646408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:26.646408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:26.646408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:26.646408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:26.646408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:26.646408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:26.646408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:26.646408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:26.646408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:26.646408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:26.646408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:26.646408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:26.646408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:26.646408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:26.646408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:26.647476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:26.647476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:26.647476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:26.647476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:26.647476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:26.647476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:26.647476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:26.647476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:26.649931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 348/700] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:26.650960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93600e-5e92-48d7-a8bb-4d8f4a52b02a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:26.676912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c3bb8-ba6c-4957-97bc-ba679a0f39bf None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:26.705467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:26.728047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483751a-4a83-4028-b087-b6c4673b2074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:26.728047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483751a-4a83-4028-b087-b6c4673b2074 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:26.728047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483751a-4a83-4028-b087-b6c4673b2074 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:26.728047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483751a-4a83-4028-b087-b6c4673b2074 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:26.728434 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483751a-4a83-4028-b087-b6c4673b2074 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:26.728434 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483751a-4a83-4028-b087-b6c4673b2074 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:26.728434 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483751a-4a83-4028-b087-b6c4673b2074 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:26.744358 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 353/701] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:46:26.744358 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483751a-4a83-4028-b087-b6c4673b2074 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:26.770140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93600e-5e92-48d7-a8bb-4d8f4a52b02a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:26.805663 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde624c-e685-4827-b711-38e6b23a55cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:26.806323 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde624c-e685-4827-b711-38e6b23a55cb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:26.806601 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde624c-e685-4827-b711-38e6b23a55cb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:26.806879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde624c-e685-4827-b711-38e6b23a55cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:26.817970 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde624c-e685-4827-b711-38e6b23a55cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:26.822085 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde624c-e685-4827-b711-38e6b23a55cb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:26.833712 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:26.838373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-9bde624c-e685-4827-b711-38e6b23a55cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:26.838373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:26.838373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:26.838373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:26.838373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:26.838373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:26.838373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:26.838373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:26.838373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:26.838373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:26.838373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:26.838373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:26.838373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:26.838373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:26.838373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:26.838373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:26.839420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:26.839420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:26.839420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:26.839420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:26.839420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:26.839420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:26.839420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:26.839420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:26.839420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:26.839420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:26.839420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:26.839420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:26.839420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:26.839420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:26.839420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:26.839420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:26.839420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:26.840335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:26.840335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:26.840335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:26.840335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 349/702] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:26.841078 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fd3cc-2f8c-4900-ae23-bd7a429c203f None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:26.841214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fd3cc-2f8c-4900-ae23-bd7a429c203f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:26.841486 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fd3cc-2f8c-4900-ae23-bd7a429c203f None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:26.845995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 350/703] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:26.845995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fd3cc-2f8c-4900-ae23-bd7a429c203f None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:26.858444 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483751a-4a83-4028-b087-b6c4673b2074 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:26.900601 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc8504-cbbc-40e4-b4df-49ce28c1a8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:26.901457 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc8504-cbbc-40e4-b4df-49ce28c1a8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:26.901945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc8504-cbbc-40e4-b4df-49ce28c1a8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:26.902362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc8504-cbbc-40e4-b4df-49ce28c1a8e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:26.903296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc8504-cbbc-40e4-b4df-49ce28c1a8e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:26.903605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc8504-cbbc-40e4-b4df-49ce28c1a8e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:26.905096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc8504-cbbc-40e4-b4df-49ce28c1a8e7 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:26.922893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 354/704] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:26.923721 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc8504-cbbc-40e4-b4df-49ce28c1a8e7 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:26.956578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:26.964412 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fd3cc-2f8c-4900-ae23-bd7a429c203f None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:27.000532 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efab80e-71fc-4384-86e0-510a1f238d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:27.001078 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efab80e-71fc-4384-86e0-510a1f238d99 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:27.001359 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efab80e-71fc-4384-86e0-510a1f238d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:27.001636 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efab80e-71fc-4384-86e0-510a1f238d99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:27.008277 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efab80e-71fc-4384-86e0-510a1f238d99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:27.009724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efab80e-71fc-4384-86e0-510a1f238d99 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:27.021075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-0efab80e-71fc-4384-86e0-510a1f238d99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:27.021075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:27.021075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:27.021075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:27.021075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:27.021075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:27.021075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:27.021075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:27.021075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:27.021075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:27.021075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:27.021075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:27.021075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:27.021075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:27.021075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:27.021973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:27.021973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:27.021973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:27.021973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:27.021973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:27.021973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:27.021973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:27.021973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:27.021973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:27.021973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:27.021973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:27.021973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:27.021973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:27.021973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:27.021973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:27.021973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:27.021973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:27.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:27.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:27.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:27.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:27.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:27.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:27.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:27.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:27.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 351/705] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:27.023570 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85d4fa-5f4d-4899-8d45-108df10e651a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:27.023639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85d4fa-5f4d-4899-8d45-108df10e651a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:27.023968 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85d4fa-5f4d-4899-8d45-108df10e651a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:27.033849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc8504-cbbc-40e4-b4df-49ce28c1a8e7 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:27.041897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ef85d4fa-5f4d-4899-8d45-108df10e651a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:27.043084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ef85d4fa-5f4d-4899-8d45-108df10e651a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:27.067847 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef98d0-babf-424b-9ed7-5036427a4b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:27.068453 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef98d0-babf-424b-9ed7-5036427a4b3a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:27.068742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef98d0-babf-424b-9ed7-5036427a4b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:27.069020 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef98d0-babf-424b-9ed7-5036427a4b3a None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:27.069665 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef98d0-babf-424b-9ed7-5036427a4b3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:27.070013 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef98d0-babf-424b-9ed7-5036427a4b3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:27.071380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef98d0-babf-424b-9ed7-5036427a4b3a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:27.072763 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:27.083490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 355/706] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:27.084356 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef98d0-babf-424b-9ed7-5036427a4b3a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:27.109402 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 352/707] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:27.110716 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea0ee7-bd08-42e4-9691-fb7a9f5a500c None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:27.110987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea0ee7-bd08-42e4-9691-fb7a9f5a500c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:27.111255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea0ee7-bd08-42e4-9691-fb7a9f5a500c None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:27.114245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 353/708] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:27.114245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea0ee7-bd08-42e4-9691-fb7a9f5a500c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:27.193775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef98d0-babf-424b-9ed7-5036427a4b3a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:27.203635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:27.220953 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea0ee7-bd08-42e4-9691-fb7a9f5a500c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:27.231953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541f8f7-ef08-4783-816e-8625ad38cbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:27.232662 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541f8f7-ef08-4783-816e-8625ad38cbe3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:27.232953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541f8f7-ef08-4783-816e-8625ad38cbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:27.233229 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541f8f7-ef08-4783-816e-8625ad38cbe3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:27.233991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541f8f7-ef08-4783-816e-8625ad38cbe3 None admin] RBAC: Adding query filter params (role.id=d1bb55f25dff4694b8675e0ef5fc4ec7, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=d1047c7d18c1466d989cd895117cd807) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:27.234306 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541f8f7-ef08-4783-816e-8625ad38cbe3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:27.235848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541f8f7-ef08-4783-816e-8625ad38cbe3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:27.242010 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 356/709] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:46:27 2022] GET /identity/v3/role_assignments?user.id=d1047c7d18c1466d989cd895117cd807&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:27.242603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541f8f7-ef08-4783-816e-8625ad38cbe3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:27.268204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354c5ff-940b-4aa9-aa08-12381e874f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:27.268749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354c5ff-940b-4aa9-aa08-12381e874f25 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:27.269030 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354c5ff-940b-4aa9-aa08-12381e874f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:27.269300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354c5ff-940b-4aa9-aa08-12381e874f25 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:27.270046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354c5ff-940b-4aa9-aa08-12381e874f25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:27.270374 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354c5ff-940b-4aa9-aa08-12381e874f25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:27.271845 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354c5ff-940b-4aa9-aa08-12381e874f25 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:27.290505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 354/710] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:27.292044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1587de2-a321-4136-9c57-ab491a2c7052 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:27.292342 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1587de2-a321-4136-9c57-ab491a2c7052 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:27.292622 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1587de2-a321-4136-9c57-ab491a2c7052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:27.313108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a1587de2-a321-4136-9c57-ab491a2c7052 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:27.314382 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a1587de2-a321-4136-9c57-ab491a2c7052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:27.326070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:27.376865 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541f8f7-ef08-4783-816e-8625ad38cbe3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:27.398060 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 355/711] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:27.399065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1587de2-a321-4136-9c57-ab491a2c7052 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:27.418740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f0658-8783-485c-8241-968b463e93cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:27.419885 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f0658-8783-485c-8241-968b463e93cd None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:27.420378 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f0658-8783-485c-8241-968b463e93cd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:27.420873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f0658-8783-485c-8241-968b463e93cd None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:27.421696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f0658-8783-485c-8241-968b463e93cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:27.423388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f0658-8783-485c-8241-968b463e93cd None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:27.486103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 357/712] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 14 11:46:27 2022] POST /identity/v3/users => generated 254 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:46:27.486969 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f0658-8783-485c-8241-968b463e93cd None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:27.492099 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:27.530509 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1587de2-a321-4136-9c57-ab491a2c7052 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:27.574397 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c55211-7473-4b3d-ac2d-49675279b9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:27.575557 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c55211-7473-4b3d-ac2d-49675279b9eb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:27.575863 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c55211-7473-4b3d-ac2d-49675279b9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:27.576137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c55211-7473-4b3d-ac2d-49675279b9eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:27.576879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c55211-7473-4b3d-ac2d-49675279b9eb None admin] RBAC: Adding query filter params (role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=38b093bc4c1d4ebcbd2a6bb16b70c029) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:27.577190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c55211-7473-4b3d-ac2d-49675279b9eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:27.578801 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c55211-7473-4b3d-ac2d-49675279b9eb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:27.586035 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 356/713] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:46:27 2022] GET /identity/v3/role_assignments?user.id=38b093bc4c1d4ebcbd2a6bb16b70c029&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:27.623603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f0658-8783-485c-8241-968b463e93cd None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:27.642521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:27.668026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f7220-3f2a-4425-9cd1-61db524ab1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:27.668754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f7220-3f2a-4425-9cd1-61db524ab1dc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:27.669039 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f7220-3f2a-4425-9cd1-61db524ab1dc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:27.669314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f7220-3f2a-4425-9cd1-61db524ab1dc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:27.670114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f7220-3f2a-4425-9cd1-61db524ab1dc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:27.671634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f7220-3f2a-4425-9cd1-61db524ab1dc None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:27.685204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-643f7220-3f2a-4425-9cd1-61db524ab1dc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 11:46:27.685204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:27.685204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:46:27.685204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:46:27.685204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:46:27.685204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:46:27.685204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:46:27.685204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:27.685204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 11:46:27.685204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:27.685204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:27.685204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:27.685204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:27.685204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:27.685204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:27.685204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:27.685204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:27.687129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:27.687129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:27.687129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:27.687129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:27.687129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:27.687129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:27.687129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:46:27.687129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:46:27.687129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:46:27.687129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:46:27.687129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:27.687129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:27.687129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:27.687129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:27.687129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:27.687129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:27.688303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:27.688303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:27.688303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:27.688303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:27.688303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:27.688303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:27.688303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:27.688303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:27.688303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:27.688303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:27.688303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:46:27.688303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:27.688303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 11:46:27.688303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:27.688303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 358/714] 10.222.0.62 () {56 vars in 1129 bytes} [Wed Sep 14 11:46:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:27.693087 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c55211-7473-4b3d-ac2d-49675279b9eb None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:27.697002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f7220-3f2a-4425-9cd1-61db524ab1dc None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:27.835015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f7220-3f2a-4425-9cd1-61db524ab1dc None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:27.863586 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c55211-7473-4b3d-ac2d-49675279b9eb None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:27.882961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b2652-e1eb-4f8b-89aa-fc29d51b2f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:27.883612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b2652-e1eb-4f8b-89aa-fc29d51b2f8b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:27.883895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b2652-e1eb-4f8b-89aa-fc29d51b2f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:27.884175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b2652-e1eb-4f8b-89aa-fc29d51b2f8b None admin] PATH_INFO: `/v3/services/network` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:27.885083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b2652-e1eb-4f8b-89aa-fc29d51b2f8b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:27.887144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b2652-e1eb-4f8b-89aa-fc29d51b2f8b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:27.899814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-414b2652-e1eb-4f8b-89aa-fc29d51b2f8b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 11:46:27.899814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:27.899814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:46:27.899814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:46:27.899814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:46:27.899814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:46:27.899814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:46:27.899814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:27.899814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 11:46:27.899814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:27.899814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:27.899814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:27.899814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:27.899814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:27.899814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:27.899814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:27.899814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:27.900931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:27.900931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:27.900931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:27.900931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:27.900931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:27.900931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:27.900931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:46:27.900931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:46:27.900931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:46:27.900931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:46:27.900931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:27.900931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:27.900931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:27.900931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:27.900931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:27.900931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:27.901788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:27.901788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:27.901788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:27.901788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:27.901788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:27.901788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:27.901788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:27.901788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:27.901788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:27.901788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:27.901788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:46:27.901788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:27.901788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 11:46:27.901788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:27.901788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 359/715] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:27 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:27.909274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b2652-e1eb-4f8b-89aa-fc29d51b2f8b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:27.913903 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41da357-09ff-4c18-9c61-0ea69dc8221f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:27.914805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41da357-09ff-4c18-9c61-0ea69dc8221f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:27.915294 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41da357-09ff-4c18-9c61-0ea69dc8221f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:27.915879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41da357-09ff-4c18-9c61-0ea69dc8221f None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:27.916742 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41da357-09ff-4c18-9c61-0ea69dc8221f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:27.917252 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41da357-09ff-4c18-9c61-0ea69dc8221f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:27.919430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41da357-09ff-4c18-9c61-0ea69dc8221f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:27.925686 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 357/716] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Sep 14 11:46:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:27.931738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41da357-09ff-4c18-9c61-0ea69dc8221f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:27.935473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:27.968191 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:28.063429 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41da357-09ff-4c18-9c61-0ea69dc8221f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:28.073646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b2652-e1eb-4f8b-89aa-fc29d51b2f8b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:28.107038 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc6371-0ea5-4fc3-8808-8a44bfa70a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:28.108935 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc6371-0ea5-4fc3-8808-8a44bfa70a08 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:28.112074 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc6371-0ea5-4fc3-8808-8a44bfa70a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:28.112581 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc6371-0ea5-4fc3-8808-8a44bfa70a08 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:28.113428 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc6371-0ea5-4fc3-8808-8a44bfa70a08 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:28.113969 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc6371-0ea5-4fc3-8808-8a44bfa70a08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:28.115640 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc6371-0ea5-4fc3-8808-8a44bfa70a08 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:28.125108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 358/717] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Sep 14 11:46:27 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:28.147456 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd609e7b-0d30-48df-aa21-0e4f2a2932e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:28.148477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd609e7b-0d30-48df-aa21-0e4f2a2932e2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:28.148975 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd609e7b-0d30-48df-aa21-0e4f2a2932e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:28.149459 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd609e7b-0d30-48df-aa21-0e4f2a2932e2 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:28.150400 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd609e7b-0d30-48df-aa21-0e4f2a2932e2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:28.150930 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd609e7b-0d30-48df-aa21-0e4f2a2932e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:28.152593 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd609e7b-0d30-48df-aa21-0e4f2a2932e2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:28.162576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 360/718] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 14 11:46:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:28.167896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc6371-0ea5-4fc3-8808-8a44bfa70a08 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:28.259791 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:28.308621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc6371-0ea5-4fc3-8808-8a44bfa70a08 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:28.358751 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a317c-b56e-4dd4-b1ec-92d33ec1b183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:28.359374 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a317c-b56e-4dd4-b1ec-92d33ec1b183 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:28.359655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a317c-b56e-4dd4-b1ec-92d33ec1b183 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:28.359928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a317c-b56e-4dd4-b1ec-92d33ec1b183 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:28.360572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a317c-b56e-4dd4-b1ec-92d33ec1b183 None admin] RBAC: Adding query filter params (type=network) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:28.360873 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a317c-b56e-4dd4-b1ec-92d33ec1b183 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:28.362265 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a317c-b56e-4dd4-b1ec-92d33ec1b183 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:28.367584 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 359/719] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 14 11:46:28 2022] GET /identity/v3/services?type=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:30.089418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f08ee0-337a-4af9-8889-c00f515bcc2f None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:30.089979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f08ee0-337a-4af9-8889-c00f515bcc2f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:30.090060 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f08ee0-337a-4af9-8889-c00f515bcc2f None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:30.091756 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 361/720] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:30.097565 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b594f-5527-476b-bf5c-a3d89e8aa57b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:30.098220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b594f-5527-476b-bf5c-a3d89e8aa57b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:30.098220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b594f-5527-476b-bf5c-a3d89e8aa57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:30.127019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-374b594f-5527-476b-bf5c-a3d89e8aa57b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:30.128273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-374b594f-5527-476b-bf5c-a3d89e8aa57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:30.223743 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 360/721] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:30.228399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec076c8-9d4a-446c-afb8-bd8b430e665d None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:30.228692 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec076c8-9d4a-446c-afb8-bd8b430e665d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:30.228977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec076c8-9d4a-446c-afb8-bd8b430e665d None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:30.230336 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 362/722] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:30.235599 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8b850-d9b9-4a43-9c57-326219347869 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:30.235899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8b850-d9b9-4a43-9c57-326219347869 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:30.236181 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8b850-d9b9-4a43-9c57-326219347869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:30.255474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b52ac-c20d-4a4b-972c-14c579947b23 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:30.256108 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b52ac-c20d-4a4b-972c-14c579947b23 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:30.256620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b52ac-c20d-4a4b-972c-14c579947b23 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:30.258437 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 363/723] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:30.260973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-52f8b850-d9b9-4a43-9c57-326219347869 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:30.262810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-52f8b850-d9b9-4a43-9c57-326219347869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:30.263917 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be56b63-f7d1-4a87-a6c8-89641d0c3d2b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:30.264414 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be56b63-f7d1-4a87-a6c8-89641d0c3d2b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:30.264917 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be56b63-f7d1-4a87-a6c8-89641d0c3d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:30.298462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9be56b63-f7d1-4a87-a6c8-89641d0c3d2b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:30.300271 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9be56b63-f7d1-4a87-a6c8-89641d0c3d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:30.342414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:30.363145 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 361/724] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:30.402417 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 364/725] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 11:46:30.575012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8b850-d9b9-4a43-9c57-326219347869 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:30.679642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be56b63-f7d1-4a87-a6c8-89641d0c3d2b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:30.696374 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8b850-d9b9-4a43-9c57-326219347869 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:30.738695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:30.742550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b10a8-080d-4ecf-b651-7a8e7aa41d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:30.743197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b10a8-080d-4ecf-b651-7a8e7aa41d2f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:30.743480 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b10a8-080d-4ecf-b651-7a8e7aa41d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:30.743752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b10a8-080d-4ecf-b651-7a8e7aa41d2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:30.755039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b10a8-080d-4ecf-b651-7a8e7aa41d2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:30.760107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d51b10a8-080d-4ecf-b651-7a8e7aa41d2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:46:30.760107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:30.760107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:30.760107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:30.760107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:30.760107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:30.760107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:30.760107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:30.760107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:30.760107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:30.760107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:30.760107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:30.760107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:30.760107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:30.760107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:30.760107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:30.761238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:30.761238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:30.761238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:30.761238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:30.761238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:30.761238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:30.761238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:30.761238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:30.761238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:30.761238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:30.761238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:30.761238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:30.761238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:30.761238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:30.761238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:30.761238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:30.762127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:30.762127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:30.762127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:30.762127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:30.762127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:30.762127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:30.762127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:30.762127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:30.762127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:30.762127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:30.762127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:46:30.762127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:30.765130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 362/726] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:46:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:30.770874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b10a8-080d-4ecf-b651-7a8e7aa41d2f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:30.819008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be56b63-f7d1-4a87-a6c8-89641d0c3d2b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:30.840326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:30.865031 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609d4ca-c663-4733-b9aa-773ebd0b98e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:30.866500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609d4ca-c663-4733-b9aa-773ebd0b98e6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:30.866970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609d4ca-c663-4733-b9aa-773ebd0b98e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:30.867476 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609d4ca-c663-4733-b9aa-773ebd0b98e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:30.886274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609d4ca-c663-4733-b9aa-773ebd0b98e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:30.891651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-4609d4ca-c663-4733-b9aa-773ebd0b98e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:46:30.891651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:30.891651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:30.891651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:30.891651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:30.891651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:30.891651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:30.891651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:30.891651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:30.891651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:30.891651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:30.891651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:30.891651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:30.891651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:30.891651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:30.891651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:30.892705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:30.892705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:30.892705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:30.892705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:30.892705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:30.892705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:30.892705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:30.892705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:30.892705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:30.892705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:30.892705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:30.892705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:30.892705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:30.892705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:30.892705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:30.892705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:30.893558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:30.893558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:30.893558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:30.893558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:30.893558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:30.893558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:30.893558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:30.893558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:30.893558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:30.893558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:30.893558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:46:30.893558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:30.896087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 365/727] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:46:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:30.897170 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609d4ca-c663-4733-b9aa-773ebd0b98e6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:30.900616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b10a8-080d-4ecf-b651-7a8e7aa41d2f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:30.948326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d61b37-67ba-461b-bd0b-c3afcda79b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:30.948880 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d61b37-67ba-461b-bd0b-c3afcda79b67 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:30.949165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d61b37-67ba-461b-bd0b-c3afcda79b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:30.949441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d61b37-67ba-461b-bd0b-c3afcda79b67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:30.964535 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d61b37-67ba-461b-bd0b-c3afcda79b67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:30.970882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-29d61b37-67ba-461b-bd0b-c3afcda79b67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:30.970882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:30.970882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:30.970882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:30.970882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:30.970882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:30.970882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:30.970882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:30.970882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:30.970882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:30.970882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:30.970882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:30.970882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:30.970882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:30.970882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:30.970882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:30.971868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:30.971868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:30.971868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:30.971868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:30.971868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:30.971868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:30.971868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:30.971868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:30.971868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:30.971868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:30.971868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:30.971868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:30.971868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:30.971868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:30.971868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:30.971868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:30.972737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:30.972737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:30.972737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:30.972737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:30.972737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:30.972737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:30.972737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:30.972737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:30.972737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:30.972737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:30.972737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:30.972737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:30.975277 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 363/728] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:30.976270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d61b37-67ba-461b-bd0b-c3afcda79b67 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:31.047754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:31.055043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609d4ca-c663-4733-b9aa-773ebd0b98e6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:31.107002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd3620-e627-4666-9a0e-64e1040efaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:31.108556 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd3620-e627-4666-9a0e-64e1040efaa4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:31.108991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd3620-e627-4666-9a0e-64e1040efaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:31.109416 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd3620-e627-4666-9a0e-64e1040efaa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:31.110163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d61b37-67ba-461b-bd0b-c3afcda79b67 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:31.110585 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd3620-e627-4666-9a0e-64e1040efaa4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:31.110665 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd3620-e627-4666-9a0e-64e1040efaa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:31.112064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd3620-e627-4666-9a0e-64e1040efaa4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:31.124480 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:31.133940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 366/729] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:46:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:31.133940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd3620-e627-4666-9a0e-64e1040efaa4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:31.151666 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c915986-4f0f-4799-8bbf-fd0f4478568f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:31.152300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c915986-4f0f-4799-8bbf-fd0f4478568f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:31.152974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c915986-4f0f-4799-8bbf-fd0f4478568f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:31.153536 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c915986-4f0f-4799-8bbf-fd0f4478568f None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:31.154348 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c915986-4f0f-4799-8bbf-fd0f4478568f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:31.154604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c915986-4f0f-4799-8bbf-fd0f4478568f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:31.156036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c915986-4f0f-4799-8bbf-fd0f4478568f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:31.166197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 364/730] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:46:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:31.166197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c915986-4f0f-4799-8bbf-fd0f4478568f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:31.239983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd3620-e627-4666-9a0e-64e1040efaa4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:31.270573 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c915986-4f0f-4799-8bbf-fd0f4478568f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:31.273149 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ff7d0-79e3-462d-8046-f552de1e8564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:31.273757 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ff7d0-79e3-462d-8046-f552de1e8564 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:31.274084 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ff7d0-79e3-462d-8046-f552de1e8564 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:31.274364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ff7d0-79e3-462d-8046-f552de1e8564 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:31.275124 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ff7d0-79e3-462d-8046-f552de1e8564 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:31.275434 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ff7d0-79e3-462d-8046-f552de1e8564 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:31.276810 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ff7d0-79e3-462d-8046-f552de1e8564 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:31.284634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 367/731] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:46:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:31.285879 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e4ce1-1aae-400d-93fa-807febf670b1 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:31.286172 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e4ce1-1aae-400d-93fa-807febf670b1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:31.286450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e4ce1-1aae-400d-93fa-807febf670b1 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:31.290025 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 368/732] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:31.290025 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e4ce1-1aae-400d-93fa-807febf670b1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:31.310471 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c02c22-7bfb-4e6d-ab6d-456a6ea28a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:31.311099 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c02c22-7bfb-4e6d-ab6d-456a6ea28a46 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:31.311390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c02c22-7bfb-4e6d-ab6d-456a6ea28a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:31.311515 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:31.311660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c02c22-7bfb-4e6d-ab6d-456a6ea28a46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:31.322369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c02c22-7bfb-4e6d-ab6d-456a6ea28a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:31.323800 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c02c22-7bfb-4e6d-ab6d-456a6ea28a46 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:31.337773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-69c02c22-7bfb-4e6d-ab6d-456a6ea28a46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:31.337773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:31.337773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:31.337773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:31.337773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:31.337773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:31.337773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:31.337773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:31.337773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:31.337773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:31.337773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:31.337773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:31.337773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:31.337773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:31.337773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:31.337773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:31.338917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:31.338917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:31.338917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:31.338917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:31.338917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:31.338917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:31.338917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:31.338917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:31.338917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:31.338917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:31.338917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:31.338917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:31.338917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:31.338917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:31.338917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:31.338917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:31.338917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:31.339805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:31.339805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:31.339805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:31.339805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 365/733] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:31.340175 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2118382-072f-45e3-a0f3-718244bee2fb None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:31.340456 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2118382-072f-45e3-a0f3-718244bee2fb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:31.340724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2118382-072f-45e3-a0f3-718244bee2fb None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:31.342263 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 366/734] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:31.342972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2118382-072f-45e3-a0f3-718244bee2fb None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:31.369040 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:31.393547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e4ce1-1aae-400d-93fa-807febf670b1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:31.427735 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3e4fc-9cb4-462e-8ce4-1600ad7e82bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:31.428357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3e4fc-9cb4-462e-8ce4-1600ad7e82bc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:31.428642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3e4fc-9cb4-462e-8ce4-1600ad7e82bc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:31.428919 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3e4fc-9cb4-462e-8ce4-1600ad7e82bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:31.439743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3e4fc-9cb4-462e-8ce4-1600ad7e82bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:31.441206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3e4fc-9cb4-462e-8ce4-1600ad7e82bc None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:31.445752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2118382-072f-45e3-a0f3-718244bee2fb None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:31.455184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-5dc3e4fc-9cb4-462e-8ce4-1600ad7e82bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:31.455184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:31.455184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:31.455184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:31.455184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:31.455184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:31.455184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:31.455184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:31.455184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:31.455184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:31.455184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:31.455184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:31.455184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:31.455184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:31.455184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:31.455184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:31.456150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:31.456150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:31.456150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:31.456150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:31.456150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:31.456150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:31.456150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:31.456150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:31.456150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:31.456150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:31.456150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:31.456150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:31.456150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:31.456150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:31.456150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:31.456150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:31.456150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:31.457052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:31.457052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:31.457052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:31.457052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 369/735] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:31.457409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834898b-a0e5-4598-b94a-06b159771d36 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:31.457770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834898b-a0e5-4598-b94a-06b159771d36 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:31.457964 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834898b-a0e5-4598-b94a-06b159771d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:31.475923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4834898b-a0e5-4598-b94a-06b159771d36 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:31.477117 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4834898b-a0e5-4598-b94a-06b159771d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:31.481701 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e139d66-0112-4524-82d2-30e55038d53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:31.482358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e139d66-0112-4524-82d2-30e55038d53b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:31.482633 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e139d66-0112-4524-82d2-30e55038d53b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:31.482900 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e139d66-0112-4524-82d2-30e55038d53b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:31.492943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e139d66-0112-4524-82d2-30e55038d53b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:31.494388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e139d66-0112-4524-82d2-30e55038d53b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:31.508896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-1e139d66-0112-4524-82d2-30e55038d53b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:31.508896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:31.508896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:31.508896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:31.508896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:31.508896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:31.508896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:31.508896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:31.508896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:31.508896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:31.508896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:31.508896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:31.508896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:31.508896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:31.508896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:31.508896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:31.510041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:31.510041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:31.510041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:31.510041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:31.510041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:31.510041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:31.510041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:31.510041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:31.510041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:31.510041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:31.510041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:31.510041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:31.510041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:31.510041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:31.510041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:31.510041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:31.510041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:31.511171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:31.511171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:31.511171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:31.511171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 367/736] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:31.511171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e139d66-0112-4524-82d2-30e55038d53b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:31.554323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 370/737] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:31.554323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fc703-a393-4a3c-bbf8-2bbcf9f95add None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:31.554323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fc703-a393-4a3c-bbf8-2bbcf9f95add None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:31.554323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fc703-a393-4a3c-bbf8-2bbcf9f95add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:31.554323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:31.565785 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e44fc703-a393-4a3c-bbf8-2bbcf9f95add None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:31.567049 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e44fc703-a393-4a3c-bbf8-2bbcf9f95add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:31.595519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:31.619135 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e139d66-0112-4524-82d2-30e55038d53b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:31.636338 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 371/738] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 11:46:31.637524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fc703-a393-4a3c-bbf8-2bbcf9f95add None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:31.668148 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69232dca-8f36-481f-b4de-918c23864ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:31.669816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69232dca-8f36-481f-b4de-918c23864ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:31.670195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69232dca-8f36-481f-b4de-918c23864ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:31.670560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69232dca-8f36-481f-b4de-918c23864ad3 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:31.671580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69232dca-8f36-481f-b4de-918c23864ad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:31.671981 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69232dca-8f36-481f-b4de-918c23864ad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:31.673626 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69232dca-8f36-481f-b4de-918c23864ad3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:31.690009 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 368/739] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:31.691135 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69232dca-8f36-481f-b4de-918c23864ad3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:31.775816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fc703-a393-4a3c-bbf8-2bbcf9f95add None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:31.811819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69232dca-8f36-481f-b4de-918c23864ad3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:31.819200 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:31.824571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d60eb-f87d-4f12-aaee-1fa7033baf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:31.825700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d60eb-f87d-4f12-aaee-1fa7033baf5c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:31.826016 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d60eb-f87d-4f12-aaee-1fa7033baf5c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:31.826289 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d60eb-f87d-4f12-aaee-1fa7033baf5c None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:31.829463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d60eb-f87d-4f12-aaee-1fa7033baf5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:31.829463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d60eb-f87d-4f12-aaee-1fa7033baf5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:31.829463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d60eb-f87d-4f12-aaee-1fa7033baf5c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:31.840364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 372/740] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:31.841643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d60eb-f87d-4f12-aaee-1fa7033baf5c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:31.858593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7af8a0-4066-4b70-a234-cc374992a82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:31.859206 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7af8a0-4066-4b70-a234-cc374992a82c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:31.859492 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7af8a0-4066-4b70-a234-cc374992a82c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:31.859770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7af8a0-4066-4b70-a234-cc374992a82c None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:31.860423 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7af8a0-4066-4b70-a234-cc374992a82c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:31.860732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7af8a0-4066-4b70-a234-cc374992a82c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:31.862124 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7af8a0-4066-4b70-a234-cc374992a82c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:31.865310 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:31.875309 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 369/741] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:31.876087 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7af8a0-4066-4b70-a234-cc374992a82c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:31.949257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d60eb-f87d-4f12-aaee-1fa7033baf5c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:31.982557 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7af8a0-4066-4b70-a234-cc374992a82c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:31.982922 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14ecdc-0904-490f-b704-47e5a0af7f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:31.983550 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14ecdc-0904-490f-b704-47e5a0af7f21 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:31.983840 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14ecdc-0904-490f-b704-47e5a0af7f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:31.984108 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14ecdc-0904-490f-b704-47e5a0af7f21 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:31.991351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14ecdc-0904-490f-b704-47e5a0af7f21 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:31.992796 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14ecdc-0904-490f-b704-47e5a0af7f21 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:32.003700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-fc14ecdc-0904-490f-b704-47e5a0af7f21 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 11:46:32.003700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:32.003700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:32.003700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:32.003700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:32.003700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:32.003700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:32.003700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:32.003700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:32.003700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:32.003700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:32.003700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:32.003700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:32.003700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:32.003700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:32.003700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:32.004835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:32.004835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:32.004835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:32.004835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:32.004835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:32.004835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:32.004835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:32.004835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:32.004835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:32.004835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:32.004835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:32.004835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:32.004835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:32.004835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:32.004835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:32.004835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:32.004835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:32.004835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:32.006267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:32.006267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:32.006267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:32.006267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:32.006267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:32.006267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:32.006267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:32.006267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:32.006267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 11:46:32.006267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:32.006267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 373/742] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:46:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:32.006267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14ecdc-0904-490f-b704-47e5a0af7f21 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:32.018659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195ba2a-e853-43ee-8b23-1c215e9d0c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:32.019263 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195ba2a-e853-43ee-8b23-1c215e9d0c5d None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:32.019544 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195ba2a-e853-43ee-8b23-1c215e9d0c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:32.019818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195ba2a-e853-43ee-8b23-1c215e9d0c5d None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:32.020476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195ba2a-e853-43ee-8b23-1c215e9d0c5d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:32.021826 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195ba2a-e853-43ee-8b23-1c215e9d0c5d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:32.061648 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:32.099569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 370/743] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 14 11:46:31 2022] POST /identity/v3/services => generated 240 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:46:32.100537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195ba2a-e853-43ee-8b23-1c215e9d0c5d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:32.113999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14ecdc-0904-490f-b704-47e5a0af7f21 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:32.161420 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7bed8-8523-4a52-aa10-e89ab27d0a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:32.162218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7bed8-8523-4a52-aa10-e89ab27d0a3c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:32.162506 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7bed8-8523-4a52-aa10-e89ab27d0a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:32.162883 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:32.163502 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7bed8-8523-4a52-aa10-e89ab27d0a3c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:32.172176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7bed8-8523-4a52-aa10-e89ab27d0a3c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:32.173695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7bed8-8523-4a52-aa10-e89ab27d0a3c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:32.186280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-5fd7bed8-8523-4a52-aa10-e89ab27d0a3c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 11:46:32.186280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:32.186280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:32.186280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:32.186280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:32.186280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:32.186280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:32.186280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:32.186280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:32.186280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:32.186280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:32.186280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:32.186280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:32.186280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:32.186280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:32.186280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:32.187488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:32.187488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:32.187488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:32.187488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:32.187488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:32.187488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:32.187488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:32.187488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:32.187488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:32.187488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:32.187488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:32.187488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:32.187488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:32.187488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:32.187488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:32.187488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:32.187488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:32.187488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:32.188465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:32.188465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:32.188465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:32.188465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:32.188465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:32.188465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:32.188465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:32.188465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:32.188465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 11:46:32.188465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:32.188465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 374/744] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Sep 14 11:46:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:32.188465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7bed8-8523-4a52-aa10-e89ab27d0a3c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:32.219856 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195ba2a-e853-43ee-8b23-1c215e9d0c5d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:32.272101 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6617d7c-191e-49bc-af4a-731d0b056132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:32.272747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6617d7c-191e-49bc-af4a-731d0b056132 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:32.273031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6617d7c-191e-49bc-af4a-731d0b056132 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:32.273307 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6617d7c-191e-49bc-af4a-731d0b056132 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:32.282734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6617d7c-191e-49bc-af4a-731d0b056132 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:32.284243 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6617d7c-191e-49bc-af4a-731d0b056132 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:32.296771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d6617d7c-191e-49bc-af4a-731d0b056132 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 11:46:32.296771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:32.296771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:32.296771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:32.296771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:32.296771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:32.296771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:32.296771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:32.296771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:32.296771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:32.296771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:32.296771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:32.296771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:32.296771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:32.296771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:32.296771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:32.297738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:32.297738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:32.297738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:32.297738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:32.297738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:32.297738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:32.297738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:32.297738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:32.297738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:32.297738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:32.297738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:32.297738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:32.297738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:32.297738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:32.297738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:32.297738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:32.297738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:32.297738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:32.298734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:32.298734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:32.298734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:32.298734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:32.298734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:32.298734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:32.298734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:32.298734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:32.298734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 11:46:32.298734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:32.298734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 371/745] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 14 11:46:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:32.299699 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6617d7c-191e-49bc-af4a-731d0b056132 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:32.322343 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7bed8-8523-4a52-aa10-e89ab27d0a3c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:32.332033 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:32.358520 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfec5d-339c-4263-8f60-0a6c4e2e114c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:32.359275 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfec5d-339c-4263-8f60-0a6c4e2e114c None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:32.359561 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfec5d-339c-4263-8f60-0a6c4e2e114c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:32.359842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfec5d-339c-4263-8f60-0a6c4e2e114c None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:32.360506 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfec5d-339c-4263-8f60-0a6c4e2e114c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:32.361912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfec5d-339c-4263-8f60-0a6c4e2e114c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:32.406125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6617d7c-191e-49bc-af4a-731d0b056132 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:32.422766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:32.430468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 375/746] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 14 11:46:32 2022] POST /identity/v3/services => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:46:32.430468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfec5d-339c-4263-8f60-0a6c4e2e114c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:32.466825 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb321b-4849-4eba-8fc1-09bcaf65f865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:32.467457 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb321b-4849-4eba-8fc1-09bcaf65f865 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:32.467740 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb321b-4849-4eba-8fc1-09bcaf65f865 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:32.468014 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb321b-4849-4eba-8fc1-09bcaf65f865 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:32.468812 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb321b-4849-4eba-8fc1-09bcaf65f865 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:32.469128 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb321b-4849-4eba-8fc1-09bcaf65f865 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:32.471144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb321b-4849-4eba-8fc1-09bcaf65f865 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:32.497891 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 372/747] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Sep 14 11:46:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:32.499002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb321b-4849-4eba-8fc1-09bcaf65f865 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:32.568537 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfec5d-339c-4263-8f60-0a6c4e2e114c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:32.611159 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4718bcab-3d44-4617-ba26-934c2753e2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:32.611905 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4718bcab-3d44-4617-ba26-934c2753e2bd None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:32.612190 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4718bcab-3d44-4617-ba26-934c2753e2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:32.612469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4718bcab-3d44-4617-ba26-934c2753e2bd None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:32.613214 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4718bcab-3d44-4617-ba26-934c2753e2bd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:32.613660 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4718bcab-3d44-4617-ba26-934c2753e2bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:32.615647 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4718bcab-3d44-4617-ba26-934c2753e2bd None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:32.625409 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb321b-4849-4eba-8fc1-09bcaf65f865 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:32.637202 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 376/748] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Sep 14 11:46:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:32.638189 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4718bcab-3d44-4617-ba26-934c2753e2bd None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:32.667000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:32.668207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6c969-fcf3-48e7-accb-9f106c952b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:32.669438 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6c969-fcf3-48e7-accb-9f106c952b55 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:32.669722 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6c969-fcf3-48e7-accb-9f106c952b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:32.670126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6c969-fcf3-48e7-accb-9f106c952b55 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:32.670786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de6c969-fcf3-48e7-accb-9f106c952b55 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:32.671095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de6c969-fcf3-48e7-accb-9f106c952b55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:32.672535 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de6c969-fcf3-48e7-accb-9f106c952b55 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:32.697523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 373/749] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Sep 14 11:46:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:32.698410 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6c969-fcf3-48e7-accb-9f106c952b55 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:32.737461 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:32.763928 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4718bcab-3d44-4617-ba26-934c2753e2bd None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:32.811666 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b3395-40dd-4131-ae2d-67d65ed320e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:32.812884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b3395-40dd-4131-ae2d-67d65ed320e5 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:32.813714 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b3395-40dd-4131-ae2d-67d65ed320e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:32.814499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b3395-40dd-4131-ae2d-67d65ed320e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:32.823518 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6c969-fcf3-48e7-accb-9f106c952b55 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:32.876905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134da85c-7f02-4c27-a26b-3992aafc4efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:32.877643 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134da85c-7f02-4c27-a26b-3992aafc4efc None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:32.878076 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134da85c-7f02-4c27-a26b-3992aafc4efc None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:32.878452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134da85c-7f02-4c27-a26b-3992aafc4efc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:32.892428 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134da85c-7f02-4c27-a26b-3992aafc4efc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:32.894902 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134da85c-7f02-4c27-a26b-3992aafc4efc None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:32.906448 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b3395-40dd-4131-ae2d-67d65ed320e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:32.908151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b3395-40dd-4131-ae2d-67d65ed320e5 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:32.913268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-134da85c-7f02-4c27-a26b-3992aafc4efc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:32.913268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:32.913268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:32.913268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:32.913268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:32.913268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:32.913268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:32.913268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:32.913268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:32.913268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:32.913268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:32.913268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:32.913268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:32.913268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:32.913268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:32.913268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:32.914235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:32.914235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:32.914235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:32.914235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:32.914235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:32.914235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:32.914235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:32.914235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:32.914235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:32.914235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:32.914235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:32.914235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:32.914235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:32.914235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:32.914235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:32.914235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:32.914235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:32.915167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:32.915167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:32.915167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:32.920423 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 374/750] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:32.921433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134da85c-7f02-4c27-a26b-3992aafc4efc None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:32.941023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:33.021608 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 377/751] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:46:32 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:46:33.022931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b3395-40dd-4131-ae2d-67d65ed320e5 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:33.023042 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:33.045975 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134da85c-7f02-4c27-a26b-3992aafc4efc None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:33.083735 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6b613-19f6-4bbd-9d46-013a777ea0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:33.084461 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6b613-19f6-4bbd-9d46-013a777ea0ba None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:33.084749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6b613-19f6-4bbd-9d46-013a777ea0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:33.085082 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6b613-19f6-4bbd-9d46-013a777ea0ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:33.138202 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b3395-40dd-4131-ae2d-67d65ed320e5 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:33.158469 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6b613-19f6-4bbd-9d46-013a777ea0ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:33.160021 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6b613-19f6-4bbd-9d46-013a777ea0ba None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:33.165888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:33.176640 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40c060-8d68-42a1-a3a7-24367d4a977e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:33.177607 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40c060-8d68-42a1-a3a7-24367d4a977e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:33.177995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40c060-8d68-42a1-a3a7-24367d4a977e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:33.178344 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40c060-8d68-42a1-a3a7-24367d4a977e None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:33.179069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40c060-8d68-42a1-a3a7-24367d4a977e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:33.179480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40c060-8d68-42a1-a3a7-24367d4a977e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:33.180953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40c060-8d68-42a1-a3a7-24367d4a977e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:33.198162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 378/752] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:33.199657 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40c060-8d68-42a1-a3a7-24367d4a977e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:33.247280 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:33.268531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 375/753] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:46:32 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:46:33.269543 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6b613-19f6-4bbd-9d46-013a777ea0ba None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:33.314795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40c060-8d68-42a1-a3a7-24367d4a977e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:33.353166 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c6aab-464f-45da-abca-5223a6b7698b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:33.354805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c6aab-464f-45da-abca-5223a6b7698b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:33.355510 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c6aab-464f-45da-abca-5223a6b7698b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:33.356339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c6aab-464f-45da-abca-5223a6b7698b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:33.368510 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c6aab-464f-45da-abca-5223a6b7698b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:33.370568 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c6aab-464f-45da-abca-5223a6b7698b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:33.376670 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6b613-19f6-4bbd-9d46-013a777ea0ba None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:33.389310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-de0c6aab-464f-45da-abca-5223a6b7698b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:33.389310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:33.389310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:33.389310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:33.389310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:33.389310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:33.389310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:33.389310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:33.389310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:33.389310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:33.389310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:33.389310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:33.389310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:33.389310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:33.389310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:33.389310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:33.390335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:33.390335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:33.390335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:33.390335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:33.390335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:33.390335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:33.390335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:33.390335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:33.390335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:33.390335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:33.390335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:33.390335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:33.390335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:33.390335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:33.390335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:33.390335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:33.390335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:33.391271 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:33.391271 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:33.391271 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:33.398001 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 379/754] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:33.399207 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c6aab-464f-45da-abca-5223a6b7698b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:33.415492 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90a189-b127-4323-a470-317354f635fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:33.416102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90a189-b127-4323-a470-317354f635fa None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:33.416384 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90a189-b127-4323-a470-317354f635fa None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:33.416660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90a189-b127-4323-a470-317354f635fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:33.423869 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90a189-b127-4323-a470-317354f635fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:33.425627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90a189-b127-4323-a470-317354f635fa None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:33.437832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-1d90a189-b127-4323-a470-317354f635fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:33.437832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:33.437832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:33.437832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:33.437832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:33.437832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:33.437832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:33.437832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:33.437832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:33.437832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:33.437832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:33.437832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:33.437832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:33.437832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:33.437832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:33.438761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:33.438761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:33.438761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:33.438761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:33.438761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:33.438761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:33.438761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:33.438761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:33.438761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:33.438761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:33.438761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:33.438761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:33.438761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:33.438761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:33.438761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:33.438761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:33.438761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:33.439655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:33.439655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:33.439655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:33.439655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:33.439655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:33.439655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:33.439655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:33.439655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:33.443892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 376/755] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:33.444589 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90a189-b127-4323-a470-317354f635fa None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:33.449056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:33.495105 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:33.529578 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c6aab-464f-45da-abca-5223a6b7698b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:33.566675 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90a189-b127-4323-a470-317354f635fa None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:33.577645 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd378aea-9b87-424e-9e13-a4fb0450c592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:33.578307 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd378aea-9b87-424e-9e13-a4fb0450c592 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:33.578592 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd378aea-9b87-424e-9e13-a4fb0450c592 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:33.578866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd378aea-9b87-424e-9e13-a4fb0450c592 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:33.592959 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd378aea-9b87-424e-9e13-a4fb0450c592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:33.594614 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd378aea-9b87-424e-9e13-a4fb0450c592 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:33.606252 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718119b-f521-4fc3-85bb-d259eac55b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:33.607168 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718119b-f521-4fc3-85bb-d259eac55b0d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:33.607650 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718119b-f521-4fc3-85bb-d259eac55b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:33.608123 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718119b-f521-4fc3-85bb-d259eac55b0d None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:33.608970 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718119b-f521-4fc3-85bb-d259eac55b0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:33.609475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718119b-f521-4fc3-85bb-d259eac55b0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:33.611098 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718119b-f521-4fc3-85bb-d259eac55b0d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:33.611815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-dd378aea-9b87-424e-9e13-a4fb0450c592 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:33.611815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:33.611815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:33.611815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:33.611815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:33.611815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:33.611815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:33.611815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:33.611815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:33.611815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:33.611815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:33.611815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:33.611815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:33.611815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:33.611815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:33.611815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:33.612722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:33.612722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:33.612722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:33.612722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:33.612722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:33.612722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:33.612722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:33.612722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:33.612722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:33.612722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:33.612722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:33.612722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:33.612722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:33.612722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:33.612722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:33.612722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:33.612722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:33.613640 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:33.613640 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:33.613640 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:33.613640 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 380/756] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:33.614271 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd378aea-9b87-424e-9e13-a4fb0450c592 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:33.724143 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:33.735675 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd378aea-9b87-424e-9e13-a4fb0450c592 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:33.763960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 377/757] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:33.763960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718119b-f521-4fc3-85bb-d259eac55b0d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:33.779357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076f7f2-1009-466c-88f2-517fd3fa8b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:33.780323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076f7f2-1009-466c-88f2-517fd3fa8b60 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:33.780906 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076f7f2-1009-466c-88f2-517fd3fa8b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:33.781386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076f7f2-1009-466c-88f2-517fd3fa8b60 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:33.782377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076f7f2-1009-466c-88f2-517fd3fa8b60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:33.782892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076f7f2-1009-466c-88f2-517fd3fa8b60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:33.784787 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076f7f2-1009-466c-88f2-517fd3fa8b60 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:33.798854 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 381/758] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:33.800235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076f7f2-1009-466c-88f2-517fd3fa8b60 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:33.872272 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:33.875692 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718119b-f521-4fc3-85bb-d259eac55b0d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:33.913322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca11683-e85a-4bfc-93eb-e8117a5393d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:33.913989 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca11683-e85a-4bfc-93eb-e8117a5393d7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:33.914275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca11683-e85a-4bfc-93eb-e8117a5393d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:33.914523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca11683-e85a-4bfc-93eb-e8117a5393d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:33.915226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca11683-e85a-4bfc-93eb-e8117a5393d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:33.915535 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca11683-e85a-4bfc-93eb-e8117a5393d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:33.917006 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca11683-e85a-4bfc-93eb-e8117a5393d7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:33.920522 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076f7f2-1009-466c-88f2-517fd3fa8b60 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:33.930017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 378/759] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:33.930930 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca11683-e85a-4bfc-93eb-e8117a5393d7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:33.959364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c49fb-28b5-4381-b951-69b7c05fe98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:33.960357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c49fb-28b5-4381-b951-69b7c05fe98b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:33.960859 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c49fb-28b5-4381-b951-69b7c05fe98b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:33.961385 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c49fb-28b5-4381-b951-69b7c05fe98b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:33.969198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c49fb-28b5-4381-b951-69b7c05fe98b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:33.970950 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c49fb-28b5-4381-b951-69b7c05fe98b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:33.983924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-7f2c49fb-28b5-4381-b951-69b7c05fe98b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:33.983924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:33.983924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:33.983924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:33.983924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:33.983924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:33.983924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:33.983924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:33.983924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:33.983924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:33.983924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:33.983924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:33.983924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:33.983924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:33.983924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:33.984868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:33.984868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:33.984868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:33.984868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:33.984868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:33.984868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:33.984868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:33.984868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:33.984868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:33.984868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:33.984868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:33.984868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:33.984868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:33.984868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:33.984868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:33.984868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:33.984868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:33.985838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:33.985838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:33.985838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:33.985838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:33.985838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:33.985838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:33.985838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:33.985838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:34.000658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 382/760] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:34.000658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c49fb-28b5-4381-b951-69b7c05fe98b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:34.008836 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:34.038622 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca11683-e85a-4bfc-93eb-e8117a5393d7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:34.079954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834253df-2f9a-4791-b6bb-744eed08b66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:34.081223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834253df-2f9a-4791-b6bb-744eed08b66d None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:34.081607 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834253df-2f9a-4791-b6bb-744eed08b66d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:34.082022 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834253df-2f9a-4791-b6bb-744eed08b66d None admin] PATH_INFO: `/v3/projects/449a5fac1acd478abea544446c08ecd9/users/38b093bc4c1d4ebcbd2a6bb16b70c029/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:34.084442 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834253df-2f9a-4791-b6bb-744eed08b66d None admin] RBAC: Authorizing `identity:create_grant(project_id=449a5fac1acd478abea544446c08ecd9, user_id=38b093bc4c1d4ebcbd2a6bb16b70c029, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:34.086696 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834253df-2f9a-4791-b6bb-744eed08b66d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:34.108549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c49fb-28b5-4381-b951-69b7c05fe98b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:34.145274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ac9c3-f6e4-4fc8-bfd7-220707aa441b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:34.146042 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ac9c3-f6e4-4fc8-bfd7-220707aa441b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:34.146289 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ac9c3-f6e4-4fc8-bfd7-220707aa441b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:34.146574 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ac9c3-f6e4-4fc8-bfd7-220707aa441b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:34.153803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 379/761] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:46:33 2022] PUT /identity/v3/projects/449a5fac1acd478abea544446c08ecd9/users/38b093bc4c1d4ebcbd2a6bb16b70c029/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:46:34.153803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834253df-2f9a-4791-b6bb-744eed08b66d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:34.162718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ac9c3-f6e4-4fc8-bfd7-220707aa441b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:34.164225 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ac9c3-f6e4-4fc8-bfd7-220707aa441b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:34.189826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-f62ac9c3-f6e4-4fc8-bfd7-220707aa441b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:34.189826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:34.189826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:34.189826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:34.189826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:34.189826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:34.189826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:34.189826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:34.189826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:34.189826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:34.189826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:34.189826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:34.189826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:34.189826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:34.189826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:34.190831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:34.190831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:34.190831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:34.190831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:34.190831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:34.190831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:34.190831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:34.190831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:34.190831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:34.190831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:34.190831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:34.190831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:34.190831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:34.190831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:34.190831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:34.190831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:34.190831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:34.191737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:34.191737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:34.191737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:34.191737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:34.191737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:34.191737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:34.191737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:34.191737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:34.192623 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 383/762] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:34.200866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ac9c3-f6e4-4fc8-bfd7-220707aa441b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:34.204688 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:34.361348 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:34.363587 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834253df-2f9a-4791-b6bb-744eed08b66d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:34.401323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ac9c3-f6e4-4fc8-bfd7-220707aa441b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:34.413095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea5301-1140-4421-ae4a-d372a6d2f685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:34.413710 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea5301-1140-4421-ae4a-d372a6d2f685 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:34.414029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea5301-1140-4421-ae4a-d372a6d2f685 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:34.414298 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea5301-1140-4421-ae4a-d372a6d2f685 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:34.415283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea5301-1140-4421-ae4a-d372a6d2f685 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:34.415595 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea5301-1140-4421-ae4a-d372a6d2f685 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:34.417958 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea5301-1140-4421-ae4a-d372a6d2f685 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:34.436331 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 380/763] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:34.437588 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba4de0-5b4c-4b55-8ef6-53d93305ae67 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:34.438741 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba4de0-5b4c-4b55-8ef6-53d93305ae67 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:34.438741 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba4de0-5b4c-4b55-8ef6-53d93305ae67 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:34.439520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 381/764] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:34.443338 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba4de0-5b4c-4b55-8ef6-53d93305ae67 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:34.448995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc6c58-2165-4e74-92aa-57c6f2d7fd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:34.448995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc6c58-2165-4e74-92aa-57c6f2d7fd48 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:34.448995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc6c58-2165-4e74-92aa-57c6f2d7fd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:34.448995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc6c58-2165-4e74-92aa-57c6f2d7fd48 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:34.449370 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc6c58-2165-4e74-92aa-57c6f2d7fd48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:34.449370 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc6c58-2165-4e74-92aa-57c6f2d7fd48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:34.450167 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc6c58-2165-4e74-92aa-57c6f2d7fd48 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:34.467101 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 384/765] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:46:34.468360 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e8e85-4960-4318-9e8d-116160d288d3 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:34.469106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e8e85-4960-4318-9e8d-116160d288d3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:34.469392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e8e85-4960-4318-9e8d-116160d288d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:34.491771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-116e8e85-4960-4318-9e8d-116160d288d3 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:34.492985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-116e8e85-4960-4318-9e8d-116160d288d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:34.540062 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:34.564862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba4de0-5b4c-4b55-8ef6-53d93305ae67 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:34.570696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 385/766] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:34.576220 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e8e85-4960-4318-9e8d-116160d288d3 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:34.604613 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e21c3-bf54-403e-aff5-23507537cbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:34.605237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e21c3-bf54-403e-aff5-23507537cbd8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:34.605519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e21c3-bf54-403e-aff5-23507537cbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:34.605844 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e21c3-bf54-403e-aff5-23507537cbd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:34.607504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e21c3-bf54-403e-aff5-23507537cbd8 None admin] RBAC: Adding query filter params (role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=d1047c7d18c1466d989cd895117cd807) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:34.607821 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e21c3-bf54-403e-aff5-23507537cbd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:34.609400 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e21c3-bf54-403e-aff5-23507537cbd8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:34.615647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 382/767] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:46:34 2022] GET /identity/v3/role_assignments?user.id=d1047c7d18c1466d989cd895117cd807&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:46:34.676288 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499dee4-0f13-4347-b0e0-63bc14a63576 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:34.676598 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499dee4-0f13-4347-b0e0-63bc14a63576 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:34.676847 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499dee4-0f13-4347-b0e0-63bc14a63576 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:34.678468 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 383/768] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:46:34.686560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07baf60-244f-4372-8cd2-900a4f6f9c2c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:34.686862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07baf60-244f-4372-8cd2-900a4f6f9c2c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:34.687124 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07baf60-244f-4372-8cd2-900a4f6f9c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:34.702271 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:34.719978 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a07baf60-244f-4372-8cd2-900a4f6f9c2c None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:34.721215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a07baf60-244f-4372-8cd2-900a4f6f9c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:34.751891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e8e85-4960-4318-9e8d-116160d288d3 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:34.805502 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c585393-9e25-4afa-b62e-e64aa255a756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:34.806700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c585393-9e25-4afa-b62e-e64aa255a756 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:34.807197 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c585393-9e25-4afa-b62e-e64aa255a756 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:34.807809 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c585393-9e25-4afa-b62e-e64aa255a756 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:34.808785 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c585393-9e25-4afa-b62e-e64aa255a756 None admin] RBAC: Adding query filter params (role.id=d1bb55f25dff4694b8675e0ef5fc4ec7, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=d76a164ed7e14d9eb2db016b68a84d9a) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:34.809366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c585393-9e25-4afa-b62e-e64aa255a756 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:34.811267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c585393-9e25-4afa-b62e-e64aa255a756 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:34.829893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 386/769] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:46:34 2022] GET /identity/v3/role_assignments?user.id=d76a164ed7e14d9eb2db016b68a84d9a&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:34.887807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 384/770] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 11:46:34.918766 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c585393-9e25-4afa-b62e-e64aa255a756 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:35.074047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c585393-9e25-4afa-b62e-e64aa255a756 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:35.114929 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:35.126334 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ea5eb-ea58-4bca-b300-cbc4ec109b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:35.126965 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ea5eb-ea58-4bca-b300-cbc4ec109b7f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:35.127245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ea5eb-ea58-4bca-b300-cbc4ec109b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:35.127521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ea5eb-ea58-4bca-b300-cbc4ec109b7f None admin] PATH_INFO: `/v3/services/network` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:35.128111 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ea5eb-ea58-4bca-b300-cbc4ec109b7f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:35.129476 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ea5eb-ea58-4bca-b300-cbc4ec109b7f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:35.143098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-db3ea5eb-ea58-4bca-b300-cbc4ec109b7f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 11:46:35.143098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:35.143098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:46:35.143098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:46:35.143098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:46:35.143098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:46:35.143098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:46:35.143098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:35.143098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 11:46:35.143098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:35.143098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:35.143098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:35.143098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:35.143098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:35.143098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:35.143098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:35.143098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:35.144409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:35.144409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:35.144409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:35.144409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:35.144409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:35.144409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:35.144409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:46:35.144409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:46:35.144409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:46:35.144409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:46:35.144409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:35.144409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:35.144409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:35.144409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:35.144409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:35.144409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:35.146067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:35.146067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:35.146067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:35.146067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:35.146067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:35.146067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:35.146067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:35.146067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:35.146067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:35.146067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:35.146067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:46:35.146067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:35.146067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 11:46:35.146067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:35.146067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 387/771] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:34 2022] GET /identity/v3/services/network => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:35.153774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07baf60-244f-4372-8cd2-900a4f6f9c2c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:35.279115 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ea5eb-ea58-4bca-b300-cbc4ec109b7f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:35.284154 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:35.329552 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07baf60-244f-4372-8cd2-900a4f6f9c2c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:35.391938 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffd72d-e7b3-4934-a0a0-e6867995a2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:35.393387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffd72d-e7b3-4934-a0a0-e6867995a2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:35.393921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffd72d-e7b3-4934-a0a0-e6867995a2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:35.394431 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffd72d-e7b3-4934-a0a0-e6867995a2a6 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:35.395257 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffd72d-e7b3-4934-a0a0-e6867995a2a6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:35.395783 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffd72d-e7b3-4934-a0a0-e6867995a2a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:35.397338 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffd72d-e7b3-4934-a0a0-e6867995a2a6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:35.411101 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 385/772] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 14 11:46:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:35.416215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffd72d-e7b3-4934-a0a0-e6867995a2a6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:35.476888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ea5eb-ea58-4bca-b300-cbc4ec109b7f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:35.532600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50850b3c-540a-40e7-a1e9-9ffc91f64ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:35.533244 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50850b3c-540a-40e7-a1e9-9ffc91f64ace None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:35.533530 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50850b3c-540a-40e7-a1e9-9ffc91f64ace None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:35.533809 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50850b3c-540a-40e7-a1e9-9ffc91f64ace None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:35.534570 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50850b3c-540a-40e7-a1e9-9ffc91f64ace None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:35.535964 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50850b3c-540a-40e7-a1e9-9ffc91f64ace None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:35.545547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-50850b3c-540a-40e7-a1e9-9ffc91f64ace None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 11:46:35.545547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:35.545547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:46:35.545547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:46:35.545547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:46:35.545547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:46:35.545547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:46:35.545547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:35.545547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 11:46:35.545547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:35.545547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:35.545547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:35.545547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:35.545547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:35.545547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:35.545547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:35.545547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:35.546604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:35.546604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:35.546604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:35.546604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:35.546604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:35.546604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:35.546604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:46:35.546604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:46:35.546604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:46:35.546604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:46:35.546604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:35.546604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:35.546604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:35.546604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:35.546604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:35.546604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:35.547438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:35.547438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:35.547438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:35.547438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:35.547438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:35.547438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:35.547438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:35.547438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:35.547438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:35.547438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:35.547438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:46:35.547438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:35.547438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 11:46:35.547438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:35.548335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 388/773] 10.222.0.62 () {56 vars in 1129 bytes} [Wed Sep 14 11:46:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:35.554991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50850b3c-540a-40e7-a1e9-9ffc91f64ace None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:35.617028 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffd72d-e7b3-4934-a0a0-e6867995a2a6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:35.666329 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:35.689674 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ae12a-266d-4da9-87d7-a72744e3412a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:35.690326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ae12a-266d-4da9-87d7-a72744e3412a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:35.690610 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ae12a-266d-4da9-87d7-a72744e3412a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:35.690887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ae12a-266d-4da9-87d7-a72744e3412a None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:35.691537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ae12a-266d-4da9-87d7-a72744e3412a None admin] RBAC: Adding query filter params (type=network) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:35.691844 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ae12a-266d-4da9-87d7-a72744e3412a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:35.693260 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ae12a-266d-4da9-87d7-a72744e3412a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:35.710048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 386/774] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 14 11:46:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:35.710998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ae12a-266d-4da9-87d7-a72744e3412a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:35.718200 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:35.766893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50850b3c-540a-40e7-a1e9-9ffc91f64ace None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:35.870282 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ae12a-266d-4da9-87d7-a72744e3412a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:35.879117 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242b9a6-1665-4771-aecf-37056fdb48f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:35.879749 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242b9a6-1665-4771-aecf-37056fdb48f3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:35.880033 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242b9a6-1665-4771-aecf-37056fdb48f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:35.880309 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242b9a6-1665-4771-aecf-37056fdb48f3 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:35.880952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242b9a6-1665-4771-aecf-37056fdb48f3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:35.881257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242b9a6-1665-4771-aecf-37056fdb48f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:35.882667 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242b9a6-1665-4771-aecf-37056fdb48f3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:35.899703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 389/775] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Sep 14 11:46:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:35.906804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242b9a6-1665-4771-aecf-37056fdb48f3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:35.910514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4a595-1027-4888-8683-539acb4a594a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:35.911195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4a595-1027-4888-8683-539acb4a594a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:35.911488 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4a595-1027-4888-8683-539acb4a594a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:35.911802 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4a595-1027-4888-8683-539acb4a594a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:35.912734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4a595-1027-4888-8683-539acb4a594a None admin] RBAC: Adding query filter params (interface=public, service_id=f536f6de8feb4806b7fa00387fd7b5c4, region_id=RegionOne) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:35.913181 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4a595-1027-4888-8683-539acb4a594a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:35.914662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4a595-1027-4888-8683-539acb4a594a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:35.926658 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 387/776] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Sep 14 11:46:35 2022] GET /identity/v3/endpoints?service_id=f536f6de8feb4806b7fa00387fd7b5c4&interface=public®ion_id=RegionOne => generated 194 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:35.932186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4a595-1027-4888-8683-539acb4a594a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:36.007386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:36.038128 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4a595-1027-4888-8683-539acb4a594a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:36.079426 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e30323-b5ce-41ed-9997-b704a600784f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:36.080045 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e30323-b5ce-41ed-9997-b704a600784f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:36.080331 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e30323-b5ce-41ed-9997-b704a600784f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:36.080612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e30323-b5ce-41ed-9997-b704a600784f None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:36.081226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e30323-b5ce-41ed-9997-b704a600784f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:36.082628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e30323-b5ce-41ed-9997-b704a600784f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:36.088372 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 388/777] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Sep 14 11:46:35 2022] GET /identity/v3/services => generated 772 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:46:36.164071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242b9a6-1665-4771-aecf-37056fdb48f3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:36.196119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:36.225196 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f15260-c772-4dd5-a0c0-e115ca464b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:36.225844 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f15260-c772-4dd5-a0c0-e115ca464b7a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:36.226161 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f15260-c772-4dd5-a0c0-e115ca464b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:36.226445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f15260-c772-4dd5-a0c0-e115ca464b7a None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:36.227104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f15260-c772-4dd5-a0c0-e115ca464b7a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:36.227412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f15260-c772-4dd5-a0c0-e115ca464b7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:36.228843 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f15260-c772-4dd5-a0c0-e115ca464b7a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:36.239638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 390/778] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Sep 14 11:46:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:36.245817 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e30323-b5ce-41ed-9997-b704a600784f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:36.372088 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e30323-b5ce-41ed-9997-b704a600784f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:36.404702 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:36.408453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf8020-8d6c-4b7f-8c38-177b1613a97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:36.409065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf8020-8d6c-4b7f-8c38-177b1613a97f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:36.409348 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf8020-8d6c-4b7f-8c38-177b1613a97f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:36.409623 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf8020-8d6c-4b7f-8c38-177b1613a97f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:36.410436 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf8020-8d6c-4b7f-8c38-177b1613a97f None admin] RBAC: Adding query filter params (interface=public, service_id=62045f0875f84bd0a52fc30420b79911, region_id=RegionOne) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:36.410766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf8020-8d6c-4b7f-8c38-177b1613a97f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:36.412209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf8020-8d6c-4b7f-8c38-177b1613a97f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:36.417613 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 389/779] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Sep 14 11:46:36 2022] GET /identity/v3/endpoints?service_id=62045f0875f84bd0a52fc30420b79911&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:36.426952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f15260-c772-4dd5-a0c0-e115ca464b7a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:36.554215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f15260-c772-4dd5-a0c0-e115ca464b7a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:36.603647 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea049bb3-66fb-4321-99a1-a9898953fc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:36.604630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea049bb3-66fb-4321-99a1-a9898953fc8e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:36.605119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea049bb3-66fb-4321-99a1-a9898953fc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:36.605598 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea049bb3-66fb-4321-99a1-a9898953fc8e None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:36.607091 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea049bb3-66fb-4321-99a1-a9898953fc8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:36.608775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea049bb3-66fb-4321-99a1-a9898953fc8e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:36.618903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 391/780] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Sep 14 11:46:36 2022] GET /identity/v3/services => generated 772 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:36.760006 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4bd01-7cdb-4438-83cd-51e234ec1d48 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:36.760364 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4bd01-7cdb-4438-83cd-51e234ec1d48 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:36.760561 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4bd01-7cdb-4438-83cd-51e234ec1d48 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:36.762858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 390/781] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:36.768358 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0cf91-5a74-4586-a25c-d62d00937a5d None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:36.768937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0cf91-5a74-4586-a25c-d62d00937a5d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:36.769419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0cf91-5a74-4586-a25c-d62d00937a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:36.794685 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9cc0cf91-5a74-4586-a25c-d62d00937a5d None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:36.796257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9cc0cf91-5a74-4586-a25c-d62d00937a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:36.866533 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:36.898771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 392/782] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:37.193121 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c96e82-d9c5-440f-adfd-02fe37ea617c None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:37.193534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c96e82-d9c5-440f-adfd-02fe37ea617c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:37.193664 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c96e82-d9c5-440f-adfd-02fe37ea617c None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:37.195373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 391/783] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:37.201405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9839baa-63eb-4b0d-9f66-12f1064d5a06 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:37.202428 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9839baa-63eb-4b0d-9f66-12f1064d5a06 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:37.202985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9839baa-63eb-4b0d-9f66-12f1064d5a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:37.231753 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b9839baa-63eb-4b0d-9f66-12f1064d5a06 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:37.233464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b9839baa-63eb-4b0d-9f66-12f1064d5a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:37.323562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b36bc-5c9f-44ec-9c93-d64a91b5eeeb None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:37.324231 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b36bc-5c9f-44ec-9c93-d64a91b5eeeb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:37.324748 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b36bc-5c9f-44ec-9c93-d64a91b5eeeb None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:37.326945 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 392/784] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:37.329781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b36bc-5c9f-44ec-9c93-d64a91b5eeeb None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:37.332341 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 393/785] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:37.339978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cdd2c-254d-42cd-bffa-f1e93307daee None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:37.340382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cdd2c-254d-42cd-bffa-f1e93307daee None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:37.340666 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cdd2c-254d-42cd-bffa-f1e93307daee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:37.369462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-be2cdd2c-254d-42cd-bffa-f1e93307daee None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:37.372622 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-be2cdd2c-254d-42cd-bffa-f1e93307daee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:37.462233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b36bc-5c9f-44ec-9c93-d64a91b5eeeb None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:37.511550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e348b7-f755-4c0e-b1e2-b6ab6da5fae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:37.512739 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 394/786] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:37.513612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e348b7-f755-4c0e-b1e2-b6ab6da5fae7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:37.514198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e348b7-f755-4c0e-b1e2-b6ab6da5fae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:37.514690 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e348b7-f755-4c0e-b1e2-b6ab6da5fae7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:37.526011 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e348b7-f755-4c0e-b1e2-b6ab6da5fae7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:37.531866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-a1e348b7-f755-4c0e-b1e2-b6ab6da5fae7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:46:37.531866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:37.531866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:37.531866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:37.531866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:37.531866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:37.531866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:37.531866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:37.531866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:37.531866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:37.531866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:37.531866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:37.531866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:37.531866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:37.531866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:37.531866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:37.532803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:37.532803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:37.532803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:37.532803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:37.532803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:37.532803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:37.532803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:37.532803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:37.532803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:37.532803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:37.532803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:37.532803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:37.532803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:37.532803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:37.532803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:37.532803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:37.533660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:37.533660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:37.533660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:37.533660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:37.533660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:37.533660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:37.533660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:37.533660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:37.533660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:37.533660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:37.533660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:46:37.533660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:37.536281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 393/787] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:46:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:37.542541 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cdd2c-254d-42cd-bffa-f1e93307daee None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:37.594666 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:37.642760 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e348b7-f755-4c0e-b1e2-b6ab6da5fae7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:37.649587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cdd2c-254d-42cd-bffa-f1e93307daee None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:37.691183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d26eb-b01d-43ed-aad8-6faf1502b5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:37.692376 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d26eb-b01d-43ed-aad8-6faf1502b5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:37.692870 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d26eb-b01d-43ed-aad8-6faf1502b5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:37.693449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d26eb-b01d-43ed-aad8-6faf1502b5c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:37.694377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d26eb-b01d-43ed-aad8-6faf1502b5c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:37.695274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d26eb-b01d-43ed-aad8-6faf1502b5c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:37.697045 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d26eb-b01d-43ed-aad8-6faf1502b5c0 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:37.707729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 395/788] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:46:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:37.716351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d26eb-b01d-43ed-aad8-6faf1502b5c0 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:37.762248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:37.827742 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e348b7-f755-4c0e-b1e2-b6ab6da5fae7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:37.874381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f54fd-7a39-4a12-b532-141db23739a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:37.875017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f54fd-7a39-4a12-b532-141db23739a7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:37.875299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f54fd-7a39-4a12-b532-141db23739a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:37.875574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f54fd-7a39-4a12-b532-141db23739a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:37.880932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:37.887677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d26eb-b01d-43ed-aad8-6faf1502b5c0 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:37.889778 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f54fd-7a39-4a12-b532-141db23739a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:37.894893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-968f54fd-7a39-4a12-b532-141db23739a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:37.894893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:37.894893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:37.894893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:37.894893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:37.894893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:37.894893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:37.894893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:37.894893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:37.894893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:37.894893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:37.894893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:37.894893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:37.894893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:37.894893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:37.894893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:37.898715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:37.898715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:37.898715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:37.898715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:37.898715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:37.898715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:37.898715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:37.898715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:37.898715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:37.898715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:37.898715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:37.898715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:37.898715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:37.898715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:37.898715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:37.898715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:37.899579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:37.899579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:37.899579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:37.899579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:37.899579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:37.899579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:37.899579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:37.899579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:37.899579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:37.899579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:37.899579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:37.899579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:37.899579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 394/789] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:37.899579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f54fd-7a39-4a12-b532-141db23739a7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:37.924385 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad07af4-c664-4857-8660-532e3c24b330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:37.925241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad07af4-c664-4857-8660-532e3c24b330 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:37.925705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad07af4-c664-4857-8660-532e3c24b330 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:37.926137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad07af4-c664-4857-8660-532e3c24b330 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:37.938671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad07af4-c664-4857-8660-532e3c24b330 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:37.940345 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad07af4-c664-4857-8660-532e3c24b330 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:37.957725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-dad07af4-c664-4857-8660-532e3c24b330 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:37.957725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:37.957725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:37.957725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:37.957725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:37.957725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:37.957725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:37.957725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:37.957725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:37.957725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:37.957725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:37.957725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:37.957725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:37.957725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:37.957725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:37.957725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:37.958956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:37.958956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:37.958956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:37.958956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:37.958956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:37.958956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:37.958956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:37.958956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:37.958956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:37.958956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:37.958956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:37.958956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:37.958956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:37.958956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:37.958956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:37.958956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:37.958956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:37.960245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:37.960245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:37.960245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:37.962608 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 396/790] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:37.962608 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad07af4-c664-4857-8660-532e3c24b330 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:38.014240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f54fd-7a39-4a12-b532-141db23739a7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:38.030059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:38.051959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a456d1-e62e-4769-92f3-1e218f3cfeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:38.052752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a456d1-e62e-4769-92f3-1e218f3cfeec None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:38.053092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a456d1-e62e-4769-92f3-1e218f3cfeec None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:38.053432 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a456d1-e62e-4769-92f3-1e218f3cfeec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:38.064992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a456d1-e62e-4769-92f3-1e218f3cfeec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:38.070059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad07af4-c664-4857-8660-532e3c24b330 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:38.070266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-63a456d1-e62e-4769-92f3-1e218f3cfeec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:46:38.070266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:38.070266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:38.070266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:38.070266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:38.070266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:38.070266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:38.070266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:38.070266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:38.070266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:38.070266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:38.070266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:38.070266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:38.070266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:38.070266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:38.070266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:38.072508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:38.072508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:38.072508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:38.072508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:38.072508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:38.072508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:38.072508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:38.072508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:38.072508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:38.072508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:38.072508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:38.072508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:38.072508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:38.072508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:38.072508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:38.072508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:38.075019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:38.075019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:38.075019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:38.075019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:38.075019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:38.075019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:38.075019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:38.075019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:38.075019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:38.075019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:38.075019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:46:38.075019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:38.075019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 395/791] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:46:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:38.075019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a456d1-e62e-4769-92f3-1e218f3cfeec None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:38.112900 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6ea24-ce8d-4763-90bc-144194512ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:38.113509 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6ea24-ce8d-4763-90bc-144194512ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:38.113792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6ea24-ce8d-4763-90bc-144194512ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:38.114100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6ea24-ce8d-4763-90bc-144194512ab9 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:38.114771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6ea24-ce8d-4763-90bc-144194512ab9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:38.115079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6ea24-ce8d-4763-90bc-144194512ab9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:38.116573 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6ea24-ce8d-4763-90bc-144194512ab9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:38.125760 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 397/792] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:46:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:38.126770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6ea24-ce8d-4763-90bc-144194512ab9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:38.140470 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:38.182457 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a456d1-e62e-4769-92f3-1e218f3cfeec None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:38.220826 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551ad58-aba8-4a39-8a48-ef5c1591a5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:38.221686 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551ad58-aba8-4a39-8a48-ef5c1591a5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:38.222159 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551ad58-aba8-4a39-8a48-ef5c1591a5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:38.222585 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551ad58-aba8-4a39-8a48-ef5c1591a5c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:38.223507 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551ad58-aba8-4a39-8a48-ef5c1591a5c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:38.223961 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551ad58-aba8-4a39-8a48-ef5c1591a5c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:38.225480 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551ad58-aba8-4a39-8a48-ef5c1591a5c2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:38.236334 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6ea24-ce8d-4763-90bc-144194512ab9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:38.244019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 396/793] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:38.245105 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551ad58-aba8-4a39-8a48-ef5c1591a5c2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:38.272276 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d75d7-9cd7-48f8-9872-da05cdb3548f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:38.272815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d75d7-9cd7-48f8-9872-da05cdb3548f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:38.273382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d75d7-9cd7-48f8-9872-da05cdb3548f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:38.273502 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d75d7-9cd7-48f8-9872-da05cdb3548f None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:38.274156 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d75d7-9cd7-48f8-9872-da05cdb3548f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:38.274516 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d75d7-9cd7-48f8-9872-da05cdb3548f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:38.275875 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d75d7-9cd7-48f8-9872-da05cdb3548f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:38.283880 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:38.291111 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 398/794] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:46:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:38.291924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d75d7-9cd7-48f8-9872-da05cdb3548f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:38.358178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551ad58-aba8-4a39-8a48-ef5c1591a5c2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:38.369012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:38.395159 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff02ac-b940-436c-bcd7-3e2affc7b3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:38.396060 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff02ac-b940-436c-bcd7-3e2affc7b3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:38.396493 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff02ac-b940-436c-bcd7-3e2affc7b3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:38.396994 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff02ac-b940-436c-bcd7-3e2affc7b3a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:38.401215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d75d7-9cd7-48f8-9872-da05cdb3548f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:38.410204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff02ac-b940-436c-bcd7-3e2affc7b3a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:38.412133 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff02ac-b940-436c-bcd7-3e2affc7b3a6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:38.428444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-60ff02ac-b940-436c-bcd7-3e2affc7b3a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:38.428444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:38.428444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:38.428444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:38.428444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:38.428444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:38.428444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:38.428444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:38.428444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:38.428444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:38.428444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:38.428444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:38.428444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:38.428444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:38.428444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:38.428444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:38.429500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:38.429500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:38.429500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:38.429500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:38.429500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:38.429500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:38.429500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:38.429500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:38.429500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:38.429500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:38.429500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:38.429500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:38.429500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:38.429500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:38.429500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:38.429500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:38.429500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:38.430596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:38.430596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:38.430596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:38.432418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 397/795] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:38.433471 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff02ac-b940-436c-bcd7-3e2affc7b3a6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:38.438913 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2e429-c92b-4d23-9fda-11973264391b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:38.439554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2e429-c92b-4d23-9fda-11973264391b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:38.439837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2e429-c92b-4d23-9fda-11973264391b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:38.440113 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2e429-c92b-4d23-9fda-11973264391b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:38.448380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2e429-c92b-4d23-9fda-11973264391b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:38.449942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2e429-c92b-4d23-9fda-11973264391b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:38.462130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2dc2e429-c92b-4d23-9fda-11973264391b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 11:46:38.462130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:38.462130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:38.462130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:38.462130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:38.462130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:38.462130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:38.462130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:38.462130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:38.462130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:38.462130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:38.462130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:38.462130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:38.462130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:38.462130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:38.462130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:38.463277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:38.463277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:38.463277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:38.463277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:38.463277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:38.463277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:38.463277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:38.463277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:38.463277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:38.463277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:38.463277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:38.463277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:38.463277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:38.463277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:38.463277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:38.463277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:38.463277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:38.463277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:38.464386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:38.464386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:38.464386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:38.464386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:38.464386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:38.464386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:38.464386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:38.464386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:38.464386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 11:46:38.464386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:38.468523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 399/796] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Sep 14 11:46:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:38.469392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2e429-c92b-4d23-9fda-11973264391b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:38.531459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:38.548287 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff02ac-b940-436c-bcd7-3e2affc7b3a6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:38.577089 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2e429-c92b-4d23-9fda-11973264391b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:38.586143 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb85cd7-36de-43cf-b8c9-de4c9300d50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:38.587081 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb85cd7-36de-43cf-b8c9-de4c9300d50b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:38.587516 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb85cd7-36de-43cf-b8c9-de4c9300d50b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:38.587972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb85cd7-36de-43cf-b8c9-de4c9300d50b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:38.601005 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb85cd7-36de-43cf-b8c9-de4c9300d50b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:38.602855 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb85cd7-36de-43cf-b8c9-de4c9300d50b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:38.611533 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:38.617386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50c703-69e4-46dc-9af2-f18e8f65da22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:38.622004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50c703-69e4-46dc-9af2-f18e8f65da22 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:38.622004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50c703-69e4-46dc-9af2-f18e8f65da22 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:38.622004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50c703-69e4-46dc-9af2-f18e8f65da22 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:38.622004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a50c703-69e4-46dc-9af2-f18e8f65da22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:38.622004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a50c703-69e4-46dc-9af2-f18e8f65da22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:38.622004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a50c703-69e4-46dc-9af2-f18e8f65da22 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:38.622835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-4fb85cd7-36de-43cf-b8c9-de4c9300d50b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:38.622835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:38.622835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:38.622835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:38.622835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:38.622835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:38.622835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:38.622835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:38.622835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:38.622835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:38.622835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:38.622835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:38.622835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:38.622835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:38.622835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:38.622835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:38.623931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:38.623931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:38.623931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:38.623931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:38.623931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:38.623931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:38.623931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:38.623931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:38.623931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:38.623931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:38.623931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:38.623931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:38.623931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:38.623931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:38.623931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:38.623931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:38.623931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:38.624984 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:38.624984 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:38.624984 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:38.626809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 398/797] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:38.627843 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb85cd7-36de-43cf-b8c9-de4c9300d50b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:38.641076 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 400/798] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:38.642094 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50c703-69e4-46dc-9af2-f18e8f65da22 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:38.738844 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb85cd7-36de-43cf-b8c9-de4c9300d50b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:38.748331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50c703-69e4-46dc-9af2-f18e8f65da22 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:38.773206 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c0ffa-92f7-4d8a-aa76-d8e657867a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:38.773820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c0ffa-92f7-4d8a-aa76-d8e657867a6a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:38.774135 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c0ffa-92f7-4d8a-aa76-d8e657867a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:38.774414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c0ffa-92f7-4d8a-aa76-d8e657867a6a None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:38.775141 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c0ffa-92f7-4d8a-aa76-d8e657867a6a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:38.775448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c0ffa-92f7-4d8a-aa76-d8e657867a6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:38.776907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c0ffa-92f7-4d8a-aa76-d8e657867a6a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:38.784717 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18529c61-7757-4f63-bc4e-0afb656149a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:38.785267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18529c61-7757-4f63-bc4e-0afb656149a2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:38.785548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18529c61-7757-4f63-bc4e-0afb656149a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:38.785816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18529c61-7757-4f63-bc4e-0afb656149a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:38.786283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:38.786497 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18529c61-7757-4f63-bc4e-0afb656149a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:38.786805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18529c61-7757-4f63-bc4e-0afb656149a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:38.788185 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18529c61-7757-4f63-bc4e-0afb656149a2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:38.796773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 399/799] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Sep 14 11:46:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:38.799529 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c0ffa-92f7-4d8a-aa76-d8e657867a6a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:38.808981 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 401/800] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:38.808981 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4faa765-24bb-462a-90eb-9b13c592af33 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:38.808981 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4faa765-24bb-462a-90eb-9b13c592af33 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:38.808981 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4faa765-24bb-462a-90eb-9b13c592af33 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:38.808981 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 402/801] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:38.808981 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4faa765-24bb-462a-90eb-9b13c592af33 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:38.872303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:38.914750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c0ffa-92f7-4d8a-aa76-d8e657867a6a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:38.927169 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4faa765-24bb-462a-90eb-9b13c592af33 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:38.953399 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86f772-7041-475c-9fec-8d518a9b3b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:38.954060 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86f772-7041-475c-9fec-8d518a9b3b6f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:38.954404 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86f772-7041-475c-9fec-8d518a9b3b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:38.954686 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86f772-7041-475c-9fec-8d518a9b3b6f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:38.962524 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86f772-7041-475c-9fec-8d518a9b3b6f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:38.963432 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26773ac-5bb2-44bd-a0ce-d16808033d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:38.963969 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86f772-7041-475c-9fec-8d518a9b3b6f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:38.964073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26773ac-5bb2-44bd-a0ce-d16808033d0f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:38.964375 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26773ac-5bb2-44bd-a0ce-d16808033d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:38.964594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26773ac-5bb2-44bd-a0ce-d16808033d0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:38.977604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-9d86f772-7041-475c-9fec-8d518a9b3b6f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 11:46:38.977604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:38.977604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:38.977604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:38.977604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:38.977604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:38.977604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:38.977604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:38.977604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:38.977604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:38.977604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:38.977604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:38.977604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:38.977604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:38.977604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:38.977604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:38.978672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:38.978672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:38.978672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:38.978672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:38.978672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:38.978672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:38.978672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:38.978672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:38.978672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:38.978672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:38.978672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:38.978672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:38.978672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:38.978672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:38.978672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:38.978672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:38.978672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:38.978672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:38.979639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:38.979639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:38.979639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:38.979639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:38.979639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:38.979639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:38.979639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:38.979639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:38.979639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 11:46:38.979639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:38.979639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 400/802] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 14 11:46:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:38.979639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86f772-7041-475c-9fec-8d518a9b3b6f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:39.040842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26773ac-5bb2-44bd-a0ce-d16808033d0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:39.041014 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:39.042749 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26773ac-5bb2-44bd-a0ce-d16808033d0f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:39.089509 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86f772-7041-475c-9fec-8d518a9b3b6f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:39.097208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:39.125355 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee7317-0d99-40a2-8f2d-a8c68f34aa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:39.126056 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee7317-0d99-40a2-8f2d-a8c68f34aa8a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:39.126336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee7317-0d99-40a2-8f2d-a8c68f34aa8a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:39.126610 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee7317-0d99-40a2-8f2d-a8c68f34aa8a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:39.135346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee7317-0d99-40a2-8f2d-a8c68f34aa8a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:39.136815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee7317-0d99-40a2-8f2d-a8c68f34aa8a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:39.150496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-daee7317-0d99-40a2-8f2d-a8c68f34aa8a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 11:46:39.150496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:39.150496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:39.150496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:39.150496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:39.150496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:39.150496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:39.150496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:39.150496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:39.150496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:39.150496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:39.150496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:39.150496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:39.150496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:39.150496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:39.150496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:39.151526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:39.151526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:39.151526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:39.151526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:39.151526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:39.151526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:39.151526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:39.151526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:39.151526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:39.151526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:39.151526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:39.151526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:39.151526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:39.151526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:39.151526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:39.151526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:39.151526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:39.151526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:39.152528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:39.152528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:39.152528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:39.152528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:39.152528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:39.152528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:39.152528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:39.152528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:39.152528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 11:46:39.152528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:39.152528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 401/803] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:46:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:39.153136 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b0134-b6e1-4516-87bf-0f9067e377be None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:39.154251 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b0134-b6e1-4516-87bf-0f9067e377be None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:39.154251 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b0134-b6e1-4516-87bf-0f9067e377be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:39.165151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 403/804] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:46:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:46:39.165151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26773ac-5bb2-44bd-a0ce-d16808033d0f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:39.175751 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-114b0134-b6e1-4516-87bf-0f9067e377be None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:39.176942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-114b0134-b6e1-4516-87bf-0f9067e377be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:39.258543 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 402/805] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:39.259393 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b0134-b6e1-4516-87bf-0f9067e377be None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:39.272742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26773ac-5bb2-44bd-a0ce-d16808033d0f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:39.309479 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f6673-0675-437f-a7be-e50cfd3bf596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:39.310691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f6673-0675-437f-a7be-e50cfd3bf596 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:39.311209 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f6673-0675-437f-a7be-e50cfd3bf596 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:39.311823 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f6673-0675-437f-a7be-e50cfd3bf596 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:39.312557 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:39.313511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f6673-0675-437f-a7be-e50cfd3bf596 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:39.313511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f6673-0675-437f-a7be-e50cfd3bf596 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:39.315524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f6673-0675-437f-a7be-e50cfd3bf596 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:39.342644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 404/806] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Sep 14 11:46:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:39.342644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f6673-0675-437f-a7be-e50cfd3bf596 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:39.357606 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:39.369522 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b0134-b6e1-4516-87bf-0f9067e377be None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:39.408775 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf1bff-13e3-4a51-b7da-09e6d4aab389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:39.409398 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf1bff-13e3-4a51-b7da-09e6d4aab389 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:39.409684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf1bff-13e3-4a51-b7da-09e6d4aab389 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:39.409990 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf1bff-13e3-4a51-b7da-09e6d4aab389 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:39.410725 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf1bff-13e3-4a51-b7da-09e6d4aab389 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:39.411036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf1bff-13e3-4a51-b7da-09e6d4aab389 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:39.412452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf1bff-13e3-4a51-b7da-09e6d4aab389 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:39.431479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 403/807] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Sep 14 11:46:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:39.432848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd9965-fd0d-44bd-ab8b-735ed6241b56 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:39.433208 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd9965-fd0d-44bd-ab8b-735ed6241b56 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:39.433556 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd9965-fd0d-44bd-ab8b-735ed6241b56 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:39.435583 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 404/808] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:39.436237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd9965-fd0d-44bd-ab8b-735ed6241b56 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:39.460533 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f6673-0675-437f-a7be-e50cfd3bf596 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:39.498580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215c5a7-410c-46aa-8b44-193cac48bdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:39.499212 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215c5a7-410c-46aa-8b44-193cac48bdf8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:39.499500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215c5a7-410c-46aa-8b44-193cac48bdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:39.499834 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215c5a7-410c-46aa-8b44-193cac48bdf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:39.510843 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215c5a7-410c-46aa-8b44-193cac48bdf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:39.512059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215c5a7-410c-46aa-8b44-193cac48bdf8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:39.526890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-e215c5a7-410c-46aa-8b44-193cac48bdf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:39.526890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:39.526890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:39.526890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:39.526890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:39.526890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:39.526890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:39.526890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:39.526890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:39.526890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:39.526890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:39.526890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:39.526890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:39.526890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:39.526890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:39.526890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:39.527849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:39.527849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:39.527849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:39.527849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:39.527849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:39.527849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:39.527849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:39.527849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:39.527849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:39.527849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:39.527849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:39.527849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:39.527849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:39.527849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:39.527849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:39.527849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:39.527849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:39.528793 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:39.528793 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:39.528793 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:39.528793 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 405/809] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:39.528793 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215c5a7-410c-46aa-8b44-193cac48bdf8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:39.545982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd9965-fd0d-44bd-ab8b-735ed6241b56 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:39.548284 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:39.581067 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ecff3-d180-4fe5-aa03-f54146489623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:39.581622 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ecff3-d180-4fe5-aa03-f54146489623 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:39.581934 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ecff3-d180-4fe5-aa03-f54146489623 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:39.582281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ecff3-d180-4fe5-aa03-f54146489623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:39.592744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ecff3-d180-4fe5-aa03-f54146489623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:39.594220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ecff3-d180-4fe5-aa03-f54146489623 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:39.594871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:39.609300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-ff1ecff3-d180-4fe5-aa03-f54146489623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:39.609300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:39.609300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:39.609300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:39.609300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:39.609300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:39.609300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:39.609300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:39.609300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:39.609300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:39.609300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:39.609300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:39.609300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:39.609300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:39.609300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:39.609300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:39.610407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:39.610407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:39.610407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:39.610407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:39.610407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:39.610407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:39.610407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:39.610407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:39.610407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:39.610407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:39.610407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:39.610407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:39.610407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:39.610407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:39.610407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:39.610407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:39.610407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:39.611335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:39.611335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:39.611335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:39.611335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 405/810] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:39.611847 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6d590-61b9-4102-8b2c-b0d230c885c8 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:39.611949 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6d590-61b9-4102-8b2c-b0d230c885c8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:39.612177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6d590-61b9-4102-8b2c-b0d230c885c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:39.630907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-79c6d590-61b9-4102-8b2c-b0d230c885c8 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:39.632108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-79c6d590-61b9-4102-8b2c-b0d230c885c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:39.638542 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215c5a7-410c-46aa-8b44-193cac48bdf8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:39.672041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f80d2-2b65-486f-9e17-d8b665ffbdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:39.672667 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f80d2-2b65-486f-9e17-d8b665ffbdc0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:39.672952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f80d2-2b65-486f-9e17-d8b665ffbdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:39.673230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f80d2-2b65-486f-9e17-d8b665ffbdc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:39.684388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f80d2-2b65-486f-9e17-d8b665ffbdc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:39.685948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f80d2-2b65-486f-9e17-d8b665ffbdc0 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:39.698090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 406/811] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:39.698894 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6d590-61b9-4102-8b2c-b0d230c885c8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:39.699741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-0d4f80d2-2b65-486f-9e17-d8b665ffbdc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:39.699741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:39.699741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:39.699741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:39.699741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:39.699741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:39.699741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:39.699741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:39.699741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:39.699741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:39.699741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:39.699741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:39.699741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:39.699741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:39.699741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:39.699741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:39.700688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:39.700688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:39.700688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:39.700688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:39.700688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:39.700688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:39.700688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:39.700688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:39.700688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:39.700688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:39.700688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:39.700688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:39.700688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:39.700688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:39.700688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:39.700688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:39.700688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:39.701633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:39.701633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:39.701633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:39.701633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 406/812] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:39.702276 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f80d2-2b65-486f-9e17-d8b665ffbdc0 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:39.801354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:39.816276 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6d590-61b9-4102-8b2c-b0d230c885c8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:39.823897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f80d2-2b65-486f-9e17-d8b665ffbdc0 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:39.838618 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:39.859274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8fb42-5162-459a-88fd-f64dd4f3f0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:39.859467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e00ac0-1b2b-4895-a3c3-eaa9d1852f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:39.859879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e00ac0-1b2b-4895-a3c3-eaa9d1852f14 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:39.859980 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8fb42-5162-459a-88fd-f64dd4f3f0ca None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:39.860174 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e00ac0-1b2b-4895-a3c3-eaa9d1852f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:39.860365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8fb42-5162-459a-88fd-f64dd4f3f0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:39.860462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e00ac0-1b2b-4895-a3c3-eaa9d1852f14 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:39.860537 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8fb42-5162-459a-88fd-f64dd4f3f0ca None admin] PATH_INFO: `/v3/services/network` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:39.861002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8fb42-5162-459a-88fd-f64dd4f3f0ca None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:39.861231 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e00ac0-1b2b-4895-a3c3-eaa9d1852f14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:39.861532 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e00ac0-1b2b-4895-a3c3-eaa9d1852f14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:39.862406 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8fb42-5162-459a-88fd-f64dd4f3f0ca None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:39.862912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e00ac0-1b2b-4895-a3c3-eaa9d1852f14 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:39.872286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-9dc8fb42-5162-459a-88fd-f64dd4f3f0ca None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 11:46:39.872286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:39.872286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:46:39.872286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:46:39.872286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:46:39.872286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:46:39.872286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:46:39.872286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:39.872286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 11:46:39.872286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:39.872286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:39.872286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:39.872286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:39.872286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:39.872286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:39.872286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:39.872286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:39.873267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:39.873267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:39.873267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:39.873267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:39.873267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:39.873267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:39.873267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:46:39.873267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:46:39.873267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:46:39.873267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:46:39.873267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:39.873267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:39.873267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:39.873267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:39.873267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:39.873267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:39.874144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:39.874144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:39.874144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:39.874144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:39.874144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:39.874144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:39.874144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:39.874144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:39.874144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:39.874144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:39.874144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:46:39.874144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:39.874144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 11:46:39.874144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:39.874144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 407/813] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:39 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:39.874984 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8fb42-5162-459a-88fd-f64dd4f3f0ca None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:39.880510 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 407/814] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:39.880954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e00ac0-1b2b-4895-a3c3-eaa9d1852f14 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:39.984575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8fb42-5162-459a-88fd-f64dd4f3f0ca None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:39.986943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e00ac0-1b2b-4895-a3c3-eaa9d1852f14 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:40.017438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2528776-550d-41ab-95c5-f66421ccf8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:40.018071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2528776-550d-41ab-95c5-f66421ccf8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:40.018353 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2528776-550d-41ab-95c5-f66421ccf8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:40.018624 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2528776-550d-41ab-95c5-f66421ccf8c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:40.019267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2528776-550d-41ab-95c5-f66421ccf8c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:40.019570 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2528776-550d-41ab-95c5-f66421ccf8c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:40.020006 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd53be-074e-4bde-a8b9-f6d97fa97fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:40.020611 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd53be-074e-4bde-a8b9-f6d97fa97fee None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:40.020896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd53be-074e-4bde-a8b9-f6d97fa97fee None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:40.020994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2528776-550d-41ab-95c5-f66421ccf8c1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:40.021173 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd53be-074e-4bde-a8b9-f6d97fa97fee None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:40.021930 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd53be-074e-4bde-a8b9-f6d97fa97fee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:40.022241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd53be-074e-4bde-a8b9-f6d97fa97fee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:40.023587 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd53be-074e-4bde-a8b9-f6d97fa97fee None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:40.032790 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 408/815] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:40.033590 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2528776-550d-41ab-95c5-f66421ccf8c1 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:40.035971 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 408/816] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:40.036768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd53be-074e-4bde-a8b9-f6d97fa97fee None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:40.046791 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:40.078236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:40.135966 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2528776-550d-41ab-95c5-f66421ccf8c1 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:40.140062 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd53be-074e-4bde-a8b9-f6d97fa97fee None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:40.169986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4cf93-1315-44ff-8819-538fc876780a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:40.170586 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4cf93-1315-44ff-8819-538fc876780a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:40.170866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4cf93-1315-44ff-8819-538fc876780a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:40.171141 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4cf93-1315-44ff-8819-538fc876780a None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:40.171762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4cf93-1315-44ff-8819-538fc876780a None admin] RBAC: Adding query filter params (name=network) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:40.172063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4cf93-1315-44ff-8819-538fc876780a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:40.173416 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4cf93-1315-44ff-8819-538fc876780a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:40.175138 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c5f67-7770-40b1-82c4-85c4e22f0156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:40.175762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c5f67-7770-40b1-82c4-85c4e22f0156 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:40.176047 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c5f67-7770-40b1-82c4-85c4e22f0156 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:40.176325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c5f67-7770-40b1-82c4-85c4e22f0156 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:40.178212 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 409/817] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 14 11:46:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:40.178979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4cf93-1315-44ff-8819-538fc876780a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:40.183207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c5f67-7770-40b1-82c4-85c4e22f0156 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:40.184639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c5f67-7770-40b1-82c4-85c4e22f0156 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:40.195997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-141c5f67-7770-40b1-82c4-85c4e22f0156 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:40.195997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:40.195997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:40.195997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:40.195997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:40.195997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:40.195997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:40.195997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:40.195997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:40.195997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:40.195997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:40.195997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:40.195997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:40.195997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:40.195997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:40.196987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:40.196987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:40.196987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:40.196987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:40.196987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:40.196987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:40.196987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:40.196987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:40.196987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:40.196987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:40.196987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:40.196987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:40.196987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:40.196987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:40.196987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:40.196987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:40.196987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:40.198473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:40.198473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:40.198473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:40.198473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:40.198473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:40.198473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:40.198473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:40.198473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:40.198473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 409/818] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:40.198473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c5f67-7770-40b1-82c4-85c4e22f0156 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:40.260754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:40.280111 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4cf93-1315-44ff-8819-538fc876780a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:40.281971 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:40.295890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c5f67-7770-40b1-82c4-85c4e22f0156 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:40.314509 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660aeb3-f0f4-4cd6-9b0e-a77a4cdfb330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:40.315140 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660aeb3-f0f4-4cd6-9b0e-a77a4cdfb330 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:40.315862 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660aeb3-f0f4-4cd6-9b0e-a77a4cdfb330 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:40.316148 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660aeb3-f0f4-4cd6-9b0e-a77a4cdfb330 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:40.316795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660aeb3-f0f4-4cd6-9b0e-a77a4cdfb330 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:40.318246 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660aeb3-f0f4-4cd6-9b0e-a77a4cdfb330 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:40.327460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-3660aeb3-f0f4-4cd6-9b0e-a77a4cdfb330 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 11:46:40.327460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:40.327460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:46:40.327460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:46:40.327460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:46:40.327460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:46:40.327460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:46:40.327460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:40.327460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 11:46:40.327460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:40.327460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:40.327460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:40.327460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:40.327460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:40.327460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:40.327460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:40.327460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:40.328524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:40.328524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:40.328524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:40.328524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:40.328524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:40.328524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:40.328524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:46:40.328524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:46:40.328524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:46:40.328524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:46:40.328524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:40.328524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:40.328524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:40.328524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:40.328524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:40.328524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:40.329387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:40.329387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:40.329387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:40.329387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:40.329387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:40.329387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:40.329387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:40.329387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:40.329387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:40.329387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:40.329387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:46:40.329387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:40.329387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 11:46:40.329387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:40.329387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 410/819] 10.222.0.62 () {56 vars in 1129 bytes} [Wed Sep 14 11:46:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:40.330813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660aeb3-f0f4-4cd6-9b0e-a77a4cdfb330 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:40.330899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485019b-1587-40db-90d2-465b920e27c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:40.331237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485019b-1587-40db-90d2-465b920e27c8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:40.331516 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485019b-1587-40db-90d2-465b920e27c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:40.331789 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485019b-1587-40db-90d2-465b920e27c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:40.338390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485019b-1587-40db-90d2-465b920e27c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:40.339803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485019b-1587-40db-90d2-465b920e27c8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:40.351899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-6485019b-1587-40db-90d2-465b920e27c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:40.351899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:40.351899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:40.351899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:40.351899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:40.351899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:40.351899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:40.351899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:40.351899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:40.351899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:40.351899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:40.351899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:40.351899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:40.351899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:40.351899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:40.353010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:40.353010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:40.353010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:40.353010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:40.353010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:40.353010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:40.353010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:40.353010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:40.353010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:40.353010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:40.353010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:40.353010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:40.353010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:40.353010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:40.353010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:40.353010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:40.353010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:40.354223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:40.354223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:40.354223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:40.354223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:40.354223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:40.354223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:40.354223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:40.354223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:40.358176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 410/820] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:40.359152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485019b-1587-40db-90d2-465b920e27c8 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:40.431788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660aeb3-f0f4-4cd6-9b0e-a77a4cdfb330 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:40.456844 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485019b-1587-40db-90d2-465b920e27c8 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:40.466424 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e55d2-b8b8-42a9-b3f3-cebd0c647f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:40.467043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e55d2-b8b8-42a9-b3f3-cebd0c647f62 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:40.467324 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e55d2-b8b8-42a9-b3f3-cebd0c647f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:40.467591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e55d2-b8b8-42a9-b3f3-cebd0c647f62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:40.473997 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647e55d2-b8b8-42a9-b3f3-cebd0c647f62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:40.475423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647e55d2-b8b8-42a9-b3f3-cebd0c647f62 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:40.482033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:40.486842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-647e55d2-b8b8-42a9-b3f3-cebd0c647f62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:40.486842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:40.486842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:40.486842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:40.486842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:40.486842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:40.486842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:40.486842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:40.486842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:40.486842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:40.486842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:40.486842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:40.486842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:40.486842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:40.486842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:40.487910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:40.487910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:40.487910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:40.487910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:40.487910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:40.487910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:40.487910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:40.487910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:40.487910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:40.487910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:40.487910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:40.487910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:40.487910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:40.487910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:40.487910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:40.487910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:40.487910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:40.489369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:40.489369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:40.489369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:40.489369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:40.489369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:40.489369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:40.489369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:40.489369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:40.489369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 411/821] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:40.489369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e55d2-b8b8-42a9-b3f3-cebd0c647f62 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:40.493491 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329b5f0-57ac-422a-99c8-ec71e8962b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:40.494140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329b5f0-57ac-422a-99c8-ec71e8962b51 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:40.494424 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329b5f0-57ac-422a-99c8-ec71e8962b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:40.494702 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329b5f0-57ac-422a-99c8-ec71e8962b51 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:40.495354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329b5f0-57ac-422a-99c8-ec71e8962b51 None admin] RBAC: Adding query filter params (type=network) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:40.495647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329b5f0-57ac-422a-99c8-ec71e8962b51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:40.497013 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329b5f0-57ac-422a-99c8-ec71e8962b51 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:40.507670 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 411/822] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 14 11:46:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:40.507670 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329b5f0-57ac-422a-99c8-ec71e8962b51 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:40.515248 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:40.588011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e55d2-b8b8-42a9-b3f3-cebd0c647f62 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:40.601484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329b5f0-57ac-422a-99c8-ec71e8962b51 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:40.622038 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab249119-361d-4ccc-adc0-01f096475d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:40.622805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab249119-361d-4ccc-adc0-01f096475d00 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:40.623091 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab249119-361d-4ccc-adc0-01f096475d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:40.623370 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab249119-361d-4ccc-adc0-01f096475d00 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:40.624062 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab249119-361d-4ccc-adc0-01f096475d00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:40.624372 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab249119-361d-4ccc-adc0-01f096475d00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:40.625804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab249119-361d-4ccc-adc0-01f096475d00 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:40.634542 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259d2b3-2288-4170-8f96-b2892e986b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:40.635142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259d2b3-2288-4170-8f96-b2892e986b2d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:40.635421 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259d2b3-2288-4170-8f96-b2892e986b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:40.635693 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259d2b3-2288-4170-8f96-b2892e986b2d None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:40.636312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259d2b3-2288-4170-8f96-b2892e986b2d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:40.636612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259d2b3-2288-4170-8f96-b2892e986b2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:40.637865 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 412/823] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:40.638118 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259d2b3-2288-4170-8f96-b2892e986b2d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:40.638697 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab249119-361d-4ccc-adc0-01f096475d00 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:40.642631 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 412/824] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Sep 14 11:46:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:40.647758 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259d2b3-2288-4170-8f96-b2892e986b2d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:40.703894 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:40.722230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:40.737426 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab249119-361d-4ccc-adc0-01f096475d00 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:40.747541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259d2b3-2288-4170-8f96-b2892e986b2d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:40.770830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684d0eb-7c92-4d17-a301-686ca11d11df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:40.771434 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684d0eb-7c92-4d17-a301-686ca11d11df None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:40.771715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684d0eb-7c92-4d17-a301-686ca11d11df None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:40.771991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684d0eb-7c92-4d17-a301-686ca11d11df None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:40.772678 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684d0eb-7c92-4d17-a301-686ca11d11df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:40.772989 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684d0eb-7c92-4d17-a301-686ca11d11df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:40.774462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684d0eb-7c92-4d17-a301-686ca11d11df None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:40.781250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34961dd-d6a1-4978-8872-6a236673b6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:40.781874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34961dd-d6a1-4978-8872-6a236673b6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:40.782164 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34961dd-d6a1-4978-8872-6a236673b6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:40.782438 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34961dd-d6a1-4978-8872-6a236673b6b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:40.783125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34961dd-d6a1-4978-8872-6a236673b6b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:40.783435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34961dd-d6a1-4978-8872-6a236673b6b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:40.784844 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34961dd-d6a1-4978-8872-6a236673b6b2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:40.786107 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 413/825] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:40.786875 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684d0eb-7c92-4d17-a301-686ca11d11df None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:40.797534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 413/826] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:40.798377 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34961dd-d6a1-4978-8872-6a236673b6b2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:40.894484 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684d0eb-7c92-4d17-a301-686ca11d11df None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:40.895785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34961dd-d6a1-4978-8872-6a236673b6b2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:40.909988 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:40.928463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4e4fe-b417-422c-85fb-aacd56397e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:40.929071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4e4fe-b417-422c-85fb-aacd56397e80 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:40.929252 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d22561-ed47-4b47-853e-f4944bf4ba35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:40.929373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4e4fe-b417-422c-85fb-aacd56397e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:40.929623 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4e4fe-b417-422c-85fb-aacd56397e80 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:40.929848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d22561-ed47-4b47-853e-f4944bf4ba35 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:40.930152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d22561-ed47-4b47-853e-f4944bf4ba35 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:40.930323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4e4fe-b417-422c-85fb-aacd56397e80 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:40.930413 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d22561-ed47-4b47-853e-f4944bf4ba35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:40.931124 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d22561-ed47-4b47-853e-f4944bf4ba35 None admin] RBAC: Adding query filter params (role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=38b093bc4c1d4ebcbd2a6bb16b70c029) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:40.931431 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d22561-ed47-4b47-853e-f4944bf4ba35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:40.931702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4e4fe-b417-422c-85fb-aacd56397e80 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:40.932874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d22561-ed47-4b47-853e-f4944bf4ba35 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:40.938954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 414/827] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:46:40 2022] GET /identity/v3/role_assignments?user.id=38b093bc4c1d4ebcbd2a6bb16b70c029&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:40.943095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d22561-ed47-4b47-853e-f4944bf4ba35 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:41.026166 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 414/828] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 11:46:40 2022] POST /identity/v3/endpoints => generated 337 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:46:41.026992 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4e4fe-b417-422c-85fb-aacd56397e80 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:41.036984 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:41.065509 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d22561-ed47-4b47-853e-f4944bf4ba35 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:41.122572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c1f89-fbc5-4ea8-a54d-d844d071f055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:41.123212 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c1f89-fbc5-4ea8-a54d-d844d071f055 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:41.123555 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c1f89-fbc5-4ea8-a54d-d844d071f055 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:41.123834 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c1f89-fbc5-4ea8-a54d-d844d071f055 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:41.124476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c1f89-fbc5-4ea8-a54d-d844d071f055 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:41.124782 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c1f89-fbc5-4ea8-a54d-d844d071f055 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:41.126225 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c1f89-fbc5-4ea8-a54d-d844d071f055 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:41.135750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 415/829] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Sep 14 11:46:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:41.136591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c1f89-fbc5-4ea8-a54d-d844d071f055 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:41.190996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:41.269717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c1f89-fbc5-4ea8-a54d-d844d071f055 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:41.304000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4e4fe-b417-422c-85fb-aacd56397e80 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:41.318321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c58fb-085e-467f-bdd1-50db3d220c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:41.319318 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c58fb-085e-467f-bdd1-50db3d220c8a None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:41.319755 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c58fb-085e-467f-bdd1-50db3d220c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:41.320031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c58fb-085e-467f-bdd1-50db3d220c8a None admin] PATH_INFO: `/v3/projects/449a5fac1acd478abea544446c08ecd9/users/d76a164ed7e14d9eb2db016b68a84d9a/roles/d1bb55f25dff4694b8675e0ef5fc4ec7` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:41.337825 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c58fb-085e-467f-bdd1-50db3d220c8a None admin] RBAC: Authorizing `identity:create_grant(project_id=449a5fac1acd478abea544446c08ecd9, user_id=d76a164ed7e14d9eb2db016b68a84d9a, role_id=d1bb55f25dff4694b8675e0ef5fc4ec7)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:41.339286 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c58fb-085e-467f-bdd1-50db3d220c8a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:41.346552 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6f85a-f7db-477c-860e-936d62285e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:41.347267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6f85a-f7db-477c-860e-936d62285e7f None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:41.349239 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6f85a-f7db-477c-860e-936d62285e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:41.349239 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6f85a-f7db-477c-860e-936d62285e7f None admin] PATH_INFO: `/v3/projects/449a5fac1acd478abea544446c08ecd9/users/d1047c7d18c1466d989cd895117cd807/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:41.349708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6f85a-f7db-477c-860e-936d62285e7f None admin] RBAC: Authorizing `identity:create_grant(project_id=449a5fac1acd478abea544446c08ecd9, user_id=d1047c7d18c1466d989cd895117cd807, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:41.351246 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6f85a-f7db-477c-860e-936d62285e7f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:41.402718 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 416/830] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:46:41 2022] PUT /identity/v3/projects/449a5fac1acd478abea544446c08ecd9/users/d76a164ed7e14d9eb2db016b68a84d9a/roles/d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:46:41.403532 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c58fb-085e-467f-bdd1-50db3d220c8a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:41.437912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 415/831] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:46:41 2022] PUT /identity/v3/projects/449a5fac1acd478abea544446c08ecd9/users/d1047c7d18c1466d989cd895117cd807/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:46:41.520452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:41.537338 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c58fb-085e-467f-bdd1-50db3d220c8a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:41.580099 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a531170-336b-402a-b6cd-6d4ca0d472d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:41.580755 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a531170-336b-402a-b6cd-6d4ca0d472d9 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:41.581118 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a531170-336b-402a-b6cd-6d4ca0d472d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:41.581463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a531170-336b-402a-b6cd-6d4ca0d472d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:41.582290 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a531170-336b-402a-b6cd-6d4ca0d472d9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:41.583964 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a531170-336b-402a-b6cd-6d4ca0d472d9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:41.661511 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 417/832] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 11:46:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:46:43.240479 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2b61e-7385-449d-b1bf-7520df205f52 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:43.241002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2b61e-7385-449d-b1bf-7520df205f52 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:43.241079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2b61e-7385-449d-b1bf-7520df205f52 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:43.242650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 416/833] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:43.248784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d99e6c-afb9-4267-9ad2-4c91332dcff1 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:43.249196 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d99e6c-afb9-4267-9ad2-4c91332dcff1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:43.249365 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d99e6c-afb9-4267-9ad2-4c91332dcff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:43.267926 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e0d99e6c-afb9-4267-9ad2-4c91332dcff1 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:43.269131 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e0d99e6c-afb9-4267-9ad2-4c91332dcff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:43.344207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 418/834] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:43.612782 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2b61e-7385-449d-b1bf-7520df205f52 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:43.613358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300a4c8-6df5-43e4-95ee-ce4d779de995 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:43.613652 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300a4c8-6df5-43e4-95ee-ce4d779de995 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:43.613970 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300a4c8-6df5-43e4-95ee-ce4d779de995 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:43.615798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 419/835] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:43.622268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f15a8-aae0-409a-aebe-acae27fe851e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:43.622585 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f15a8-aae0-409a-aebe-acae27fe851e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:43.622869 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f15a8-aae0-409a-aebe-acae27fe851e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:43.648946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-437f15a8-aae0-409a-aebe-acae27fe851e None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:43.650295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-437f15a8-aae0-409a-aebe-acae27fe851e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:43.690490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:43.733137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:43.733412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2b61e-7385-449d-b1bf-7520df205f52 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:43.740225 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 420/836] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 11:46:43.744885 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1f66e-1f3d-4bdb-a3ac-e6949a20963f None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:43.745218 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1f66e-1f3d-4bdb-a3ac-e6949a20963f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:43.745476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1f66e-1f3d-4bdb-a3ac-e6949a20963f None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:43.748573 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 421/837] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:46:43.754576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23311bb4-a8d0-461a-9e94-28e3d22cf6da None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:43.754859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23311bb4-a8d0-461a-9e94-28e3d22cf6da None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:43.755144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23311bb4-a8d0-461a-9e94-28e3d22cf6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:43.782448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-23311bb4-a8d0-461a-9e94-28e3d22cf6da None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:43.784237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-23311bb4-a8d0-461a-9e94-28e3d22cf6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:43.785219 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f93cb5-9c1b-496d-bc19-fcdb33d1aad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:43.785937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f93cb5-9c1b-496d-bc19-fcdb33d1aad3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:43.786224 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f93cb5-9c1b-496d-bc19-fcdb33d1aad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:43.786501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f93cb5-9c1b-496d-bc19-fcdb33d1aad3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:43.787858 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f93cb5-9c1b-496d-bc19-fcdb33d1aad3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:43.789319 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f93cb5-9c1b-496d-bc19-fcdb33d1aad3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:43.798736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-e8f93cb5-9c1b-496d-bc19-fcdb33d1aad3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 11:46:43.798736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:43.798736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:46:43.798736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:46:43.798736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:46:43.798736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:46:43.798736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:46:43.798736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:43.798736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 11:46:43.798736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:43.798736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:43.798736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:43.798736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:43.798736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:43.798736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:43.798736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:43.800087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:43.800087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:43.800087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:43.800087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:43.800087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:43.800087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:43.800087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:43.800087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:46:43.800087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:46:43.800087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:46:43.800087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:46:43.800087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:43.800087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:43.800087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:43.800087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:43.800087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:43.800087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:43.800990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:43.800990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:43.800990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:43.800990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:43.800990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:43.800990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:43.800990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:43.800990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:43.800990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:43.800990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:43.800990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:46:43.800990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:43.800990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 11:46:43.800990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:43.800990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 417/838] 10.222.0.62 () {56 vars in 1132 bytes} [Wed Sep 14 11:46:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:43.805977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f93cb5-9c1b-496d-bc19-fcdb33d1aad3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:43.882454 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 422/839] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 11:46:43.884052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a5ec7-e237-4dfe-8e9f-e3d01246799d None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:43.884447 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a5ec7-e237-4dfe-8e9f-e3d01246799d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:43.884828 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a5ec7-e237-4dfe-8e9f-e3d01246799d None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:43.887431 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 423/840] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:43.899608 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1b1e6-c1f4-46f5-829f-016e642ef96a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:43.900146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1b1e6-c1f4-46f5-829f-016e642ef96a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:43.900890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1b1e6-c1f4-46f5-829f-016e642ef96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:43.924215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-28d1b1e6-c1f4-46f5-829f-016e642ef96a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:43.925576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-28d1b1e6-c1f4-46f5-829f-016e642ef96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:43.925967 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f93cb5-9c1b-496d-bc19-fcdb33d1aad3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:43.952755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:43.962749 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ca28e-4303-49ae-82f3-a01765340ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:43.965451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ca28e-4303-49ae-82f3-a01765340ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:43.965451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ca28e-4303-49ae-82f3-a01765340ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:43.965451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ca28e-4303-49ae-82f3-a01765340ea3 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:43.965451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ca28e-4303-49ae-82f3-a01765340ea3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:43.965451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ca28e-4303-49ae-82f3-a01765340ea3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:43.967142 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ca28e-4303-49ae-82f3-a01765340ea3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:43.976651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 418/841] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 14 11:46:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:46:43.978249 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ca28e-4303-49ae-82f3-a01765340ea3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:44.013995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 424/842] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:44.029653 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1b1e6-c1f4-46f5-829f-016e642ef96a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:44.089713 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:44.099220 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ca28e-4303-49ae-82f3-a01765340ea3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:44.132049 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1b1e6-c1f4-46f5-829f-016e642ef96a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:44.136363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248323a-8e41-414b-86d1-fac45f24645e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:44.137478 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248323a-8e41-414b-86d1-fac45f24645e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:44.138030 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248323a-8e41-414b-86d1-fac45f24645e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:44.138542 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248323a-8e41-414b-86d1-fac45f24645e None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:44.139468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248323a-8e41-414b-86d1-fac45f24645e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:44.139972 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248323a-8e41-414b-86d1-fac45f24645e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:44.141653 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248323a-8e41-414b-86d1-fac45f24645e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:44.147162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 419/843] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 14 11:46:43 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:44.153250 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248323a-8e41-414b-86d1-fac45f24645e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:44.187791 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e957d-4579-4ef1-a8c2-ba4691483012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:44.188456 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e957d-4579-4ef1-a8c2-ba4691483012 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:44.188808 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e957d-4579-4ef1-a8c2-ba4691483012 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:44.189093 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e957d-4579-4ef1-a8c2-ba4691483012 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:44.204114 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e957d-4579-4ef1-a8c2-ba4691483012 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:44.211046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-fd4e957d-4579-4ef1-a8c2-ba4691483012 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:44.211046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:44.211046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:44.211046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:44.211046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:44.211046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:44.211046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:44.211046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:44.211046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:44.211046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:44.211046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:44.211046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:44.211046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:44.211046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:44.211046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:44.211046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:44.212119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:44.212119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:44.212119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:44.212119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:44.212119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:44.212119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:44.212119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:44.212119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:44.212119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:44.212119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:44.212119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:44.212119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:44.212119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:44.212119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:44.212119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:44.212119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:44.212954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:44.212954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:44.212954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:44.212954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:44.212954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:44.212954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:44.212954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:44.212954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:44.212954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:44.212954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:44.212954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:46:44.212954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:44.212954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 425/844] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:46:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:44.221030 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e957d-4579-4ef1-a8c2-ba4691483012 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:44.228910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:44.281148 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248323a-8e41-414b-86d1-fac45f24645e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:44.326493 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5172ff-30a6-4319-9919-f131cf4ba1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:44.327173 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5172ff-30a6-4319-9919-f131cf4ba1ad None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:44.327430 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5172ff-30a6-4319-9919-f131cf4ba1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:44.327717 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5172ff-30a6-4319-9919-f131cf4ba1ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:44.339692 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5172ff-30a6-4319-9919-f131cf4ba1ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:44.345671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-9c5172ff-30a6-4319-9919-f131cf4ba1ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:46:44.345671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:44.345671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:44.345671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:44.345671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:44.345671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:44.345671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:44.345671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:44.345671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:44.345671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:44.345671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:44.345671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:44.345671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:44.345671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:44.345671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:44.345671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:44.346646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:44.346646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:44.346646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:44.346646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:44.346646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:44.346646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:44.346646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:44.346646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:44.346646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:44.346646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:44.346646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:44.346646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:44.346646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:44.346646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:44.346646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:44.346646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:44.350813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:44.350813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:44.350813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:44.350813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:44.350813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:44.350813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:44.350813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:44.350813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:44.350813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:44.350813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:44.350813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:46:44.350813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:44.350813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 420/845] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:46:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:44.350813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5172ff-30a6-4319-9919-f131cf4ba1ad None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:44.354888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e957d-4579-4ef1-a8c2-ba4691483012 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:44.381203 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:44.404268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d75e9-631b-45d6-b610-8bad3165b3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:44.404268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d75e9-631b-45d6-b610-8bad3165b3ed None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:44.404268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d75e9-631b-45d6-b610-8bad3165b3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:44.404268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d75e9-631b-45d6-b610-8bad3165b3ed None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:44.404656 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d75e9-631b-45d6-b610-8bad3165b3ed None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:44.404656 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d75e9-631b-45d6-b610-8bad3165b3ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:44.404656 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d75e9-631b-45d6-b610-8bad3165b3ed None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:44.408271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 426/846] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:46:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:46:44.408855 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d75e9-631b-45d6-b610-8bad3165b3ed None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:44.462731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5172ff-30a6-4319-9919-f131cf4ba1ad None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:44.477220 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:44.497485 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98188a67-3fd0-4d53-858d-5436f539f799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:44.498214 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98188a67-3fd0-4d53-858d-5436f539f799 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:44.498667 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98188a67-3fd0-4d53-858d-5436f539f799 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:44.499286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98188a67-3fd0-4d53-858d-5436f539f799 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:44.499884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98188a67-3fd0-4d53-858d-5436f539f799 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:44.501655 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98188a67-3fd0-4d53-858d-5436f539f799 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:44.510477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-98188a67-3fd0-4d53-858d-5436f539f799 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 11:46:44.510477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:44.510477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:46:44.510477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:46:44.510477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:46:44.510477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:46:44.510477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:46:44.510477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:44.510477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 11:46:44.510477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:44.510477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:44.510477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:44.510477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:44.510477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:44.510477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:44.510477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:44.510477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:44.511467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:44.511467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:44.511467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:44.511467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:44.511467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:44.511467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:44.511467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:46:44.511467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:46:44.511467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:46:44.511467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:46:44.511467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:44.511467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:44.511467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:44.511467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:44.511467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:44.511467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:44.512362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:44.512362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:44.512362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:44.512362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:44.512362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:44.512362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:44.512362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:44.512362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:44.512362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:44.512362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:44.512362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:46:44.512362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:44.512362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 11:46:44.512362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:44.512362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 421/847] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:46:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:44.513252 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98188a67-3fd0-4d53-858d-5436f539f799 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:44.523807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d75e9-631b-45d6-b610-8bad3165b3ed None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:44.559766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56552b-0cab-4eb6-96cc-727aec42b943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:44.560450 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56552b-0cab-4eb6-96cc-727aec42b943 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:44.560742 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56552b-0cab-4eb6-96cc-727aec42b943 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:44.561019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56552b-0cab-4eb6-96cc-727aec42b943 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:44.561684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56552b-0cab-4eb6-96cc-727aec42b943 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:44.562022 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56552b-0cab-4eb6-96cc-727aec42b943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:44.563392 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56552b-0cab-4eb6-96cc-727aec42b943 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:44.577151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 427/848] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:46:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:44.577151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56552b-0cab-4eb6-96cc-727aec42b943 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:44.624075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:44.631042 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98188a67-3fd0-4d53-858d-5436f539f799 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:44.667459 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a6c79-ab0f-45b0-b2bd-4b1fa7957915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:44.668549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a6c79-ab0f-45b0-b2bd-4b1fa7957915 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:44.668853 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a6c79-ab0f-45b0-b2bd-4b1fa7957915 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:44.669129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a6c79-ab0f-45b0-b2bd-4b1fa7957915 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:44.680531 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a6c79-ab0f-45b0-b2bd-4b1fa7957915 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:44.682002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a6c79-ab0f-45b0-b2bd-4b1fa7957915 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:44.689383 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56552b-0cab-4eb6-96cc-727aec42b943 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:44.697382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-984a6c79-ab0f-45b0-b2bd-4b1fa7957915 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:44.697382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:44.697382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:44.697382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:44.697382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:44.697382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:44.697382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:44.697382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:44.697382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:44.697382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:44.697382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:44.697382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:44.697382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:44.697382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:44.697382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:44.697382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:44.698401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:44.698401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:44.698401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:44.698401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:44.698401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:44.698401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:44.698401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:44.698401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:44.698401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:44.698401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:44.698401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:44.698401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:44.698401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:44.698401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:44.698401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:44.698401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:44.698401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:44.699931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:44.699931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:44.699931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:44.699931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 422/849] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:44.699931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a6c79-ab0f-45b0-b2bd-4b1fa7957915 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:44.730893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02553e-1c60-45bb-ad61-c286d84e76cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:44.731339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:44.731537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02553e-1c60-45bb-ad61-c286d84e76cf None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:44.731787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02553e-1c60-45bb-ad61-c286d84e76cf None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:44.732060 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02553e-1c60-45bb-ad61-c286d84e76cf None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:44.732685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02553e-1c60-45bb-ad61-c286d84e76cf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:44.733118 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02553e-1c60-45bb-ad61-c286d84e76cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:44.734390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02553e-1c60-45bb-ad61-c286d84e76cf None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:44.744249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 428/850] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:46:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:44.744249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02553e-1c60-45bb-ad61-c286d84e76cf None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:44.810148 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a6c79-ab0f-45b0-b2bd-4b1fa7957915 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:44.853749 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a85ff-0bab-4d9b-94ec-257adf9557dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:44.854410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a85ff-0bab-4d9b-94ec-257adf9557dd None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:44.854659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02553e-1c60-45bb-ad61-c286d84e76cf None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:44.854762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a85ff-0bab-4d9b-94ec-257adf9557dd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:44.854952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a85ff-0bab-4d9b-94ec-257adf9557dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:44.864669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:44.867844 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a85ff-0bab-4d9b-94ec-257adf9557dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:44.869300 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a85ff-0bab-4d9b-94ec-257adf9557dd None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:44.884114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-231a85ff-0bab-4d9b-94ec-257adf9557dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:44.884114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:44.884114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:44.884114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:44.884114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:44.884114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:44.884114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:44.884114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:44.884114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:44.884114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:44.884114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:44.884114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:44.884114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:44.884114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:44.884114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:44.884114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:44.891056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:44.891056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:44.891056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:44.891056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:44.891056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:44.891056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:44.891056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:44.891056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:44.891056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:44.891056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:44.891056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:44.891056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:44.891056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:44.891056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:44.891056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:44.891056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:44.891056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:44.891988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:44.891988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:44.891988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:44.891988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 423/851] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:44.891988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a85ff-0bab-4d9b-94ec-257adf9557dd None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:44.896795 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d1a3c-152f-408d-8c40-4b25a22bc3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:44.897419 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d1a3c-152f-408d-8c40-4b25a22bc3be None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:44.897696 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d1a3c-152f-408d-8c40-4b25a22bc3be None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:44.898038 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d1a3c-152f-408d-8c40-4b25a22bc3be None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:44.898805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d1a3c-152f-408d-8c40-4b25a22bc3be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:44.899120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d1a3c-152f-408d-8c40-4b25a22bc3be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:44.900639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d1a3c-152f-408d-8c40-4b25a22bc3be None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:44.920133 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 429/852] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:44.920962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d1a3c-152f-408d-8c40-4b25a22bc3be None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:44.975240 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:44.993558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a85ff-0bab-4d9b-94ec-257adf9557dd None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:45.029501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee4290-67f8-4149-a619-a4ae3429a63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:45.030243 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4290-67f8-4149-a619-a4ae3429a63d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:45.030532 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4290-67f8-4149-a619-a4ae3429a63d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:45.030810 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4290-67f8-4149-a619-a4ae3429a63d None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:45.031446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee4290-67f8-4149-a619-a4ae3429a63d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:45.031755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee4290-67f8-4149-a619-a4ae3429a63d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:45.032849 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d1a3c-152f-408d-8c40-4b25a22bc3be None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:45.033152 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee4290-67f8-4149-a619-a4ae3429a63d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:45.046199 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 424/853] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:46:44 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:45.046199 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee4290-67f8-4149-a619-a4ae3429a63d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:45.083563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922671c3-a689-4b1b-a22f-809364cfcfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:45.084727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922671c3-a689-4b1b-a22f-809364cfcfeb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:45.085363 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922671c3-a689-4b1b-a22f-809364cfcfeb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:45.085713 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922671c3-a689-4b1b-a22f-809364cfcfeb None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:45.086505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922671c3-a689-4b1b-a22f-809364cfcfeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:45.086882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922671c3-a689-4b1b-a22f-809364cfcfeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:45.088628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922671c3-a689-4b1b-a22f-809364cfcfeb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:45.102698 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 430/854] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:45.108052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922671c3-a689-4b1b-a22f-809364cfcfeb None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:45.142948 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:45.180932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee4290-67f8-4149-a619-a4ae3429a63d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:45.228832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922671c3-a689-4b1b-a22f-809364cfcfeb None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:45.238413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03704b1-61b4-4782-92fa-a58790948bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:45.239060 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03704b1-61b4-4782-92fa-a58790948bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:45.239341 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03704b1-61b4-4782-92fa-a58790948bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:45.239614 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03704b1-61b4-4782-92fa-a58790948bf3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:45.253613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03704b1-61b4-4782-92fa-a58790948bf3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:45.255822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03704b1-61b4-4782-92fa-a58790948bf3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:45.270026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-d03704b1-61b4-4782-92fa-a58790948bf3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 11:46:45.270026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:45.270026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:45.270026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:45.270026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:45.270026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:45.270026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:45.270026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:45.270026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:45.270026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:45.270026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:45.270026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:45.270026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:45.270026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:45.270026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:45.270026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:45.271041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:45.271041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:45.271041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:45.271041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:45.271041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:45.271041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:45.271041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:45.271041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:45.271041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:45.271041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:45.271041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:45.271041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:45.271041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:45.271041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:45.271041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:45.271041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:45.271041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:45.271041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:45.272131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:45.272131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:45.272131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:45.272131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:45.272131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:45.272131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:45.272131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:45.272131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:45.272131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 11:46:45.272131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:45.276068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 425/855] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 14 11:46:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:45.277388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03704b1-61b4-4782-92fa-a58790948bf3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:45.281144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95405f-4b6a-46e0-8978-6a70c080e5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:45.282032 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95405f-4b6a-46e0-8978-6a70c080e5cc None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:45.282502 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95405f-4b6a-46e0-8978-6a70c080e5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:45.282961 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95405f-4b6a-46e0-8978-6a70c080e5cc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:45.287052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:45.292847 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95405f-4b6a-46e0-8978-6a70c080e5cc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:45.294719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95405f-4b6a-46e0-8978-6a70c080e5cc None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:45.307528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-2e95405f-4b6a-46e0-8978-6a70c080e5cc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 11:46:45.307528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:45.307528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:45.307528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:45.307528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:45.307528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:45.307528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:45.307528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:45.307528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:45.307528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:45.307528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:45.307528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:45.307528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:45.307528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:45.307528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:45.307528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:45.309750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:45.309750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:45.309750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:45.309750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:45.309750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:45.309750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:45.309750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:45.309750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:45.309750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:45.309750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:45.309750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:45.309750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:45.309750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:45.309750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:45.309750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:45.309750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:45.309750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:45.309750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:45.310764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:45.310764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:45.310764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:45.310764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:45.310764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:45.310764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:45.310764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:45.310764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:45.310764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 11:46:45.310764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:45.312921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 431/856] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:46:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:45.318888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95405f-4b6a-46e0-8978-6a70c080e5cc None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:45.388866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03704b1-61b4-4782-92fa-a58790948bf3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:45.425927 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a30ba-7b5f-48b9-a594-96f196d19c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:45.426853 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a30ba-7b5f-48b9-a594-96f196d19c3b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:45.427322 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a30ba-7b5f-48b9-a594-96f196d19c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:45.427825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a30ba-7b5f-48b9-a594-96f196d19c3b None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:45.428769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a30ba-7b5f-48b9-a594-96f196d19c3b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:45.429261 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a30ba-7b5f-48b9-a594-96f196d19c3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:45.430912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a30ba-7b5f-48b9-a594-96f196d19c3b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:45.442635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:45.448543 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95405f-4b6a-46e0-8978-6a70c080e5cc None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:45.452269 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 426/857] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Sep 14 11:46:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:45.459093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a30ba-7b5f-48b9-a594-96f196d19c3b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:45.490989 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444a12a-d326-4e36-b7ca-6bb7561ae4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:45.492127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444a12a-d326-4e36-b7ca-6bb7561ae4b8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:45.492477 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444a12a-d326-4e36-b7ca-6bb7561ae4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:45.492953 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444a12a-d326-4e36-b7ca-6bb7561ae4b8 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:45.493917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444a12a-d326-4e36-b7ca-6bb7561ae4b8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:45.494304 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444a12a-d326-4e36-b7ca-6bb7561ae4b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:45.496111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444a12a-d326-4e36-b7ca-6bb7561ae4b8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:45.522848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 432/858] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Sep 14 11:46:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:46:45.524199 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444a12a-d326-4e36-b7ca-6bb7561ae4b8 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:45.537997 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:45.579193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a30ba-7b5f-48b9-a594-96f196d19c3b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:45.623385 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f03a84-66ec-43bc-b6a5-7a8ebd4bb365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:45.624025 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f03a84-66ec-43bc-b6a5-7a8ebd4bb365 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:45.624311 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f03a84-66ec-43bc-b6a5-7a8ebd4bb365 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:45.624598 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f03a84-66ec-43bc-b6a5-7a8ebd4bb365 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:45.659541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444a12a-d326-4e36-b7ca-6bb7561ae4b8 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:45.706394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a70c0-3429-447f-aa08-fcd0f1cdfdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:45.707920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a70c0-3429-447f-aa08-fcd0f1cdfdf7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:45.708703 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a70c0-3429-447f-aa08-fcd0f1cdfdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:45.709247 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a70c0-3429-447f-aa08-fcd0f1cdfdf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:45.709354 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f03a84-66ec-43bc-b6a5-7a8ebd4bb365 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:45.711055 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f03a84-66ec-43bc-b6a5-7a8ebd4bb365 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:45.731616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:45.792044 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:45.803325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a70c0-3429-447f-aa08-fcd0f1cdfdf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:45.805851 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a70c0-3429-447f-aa08-fcd0f1cdfdf7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:45.846019 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 427/859] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:46:45 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:46:45.849109 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f03a84-66ec-43bc-b6a5-7a8ebd4bb365 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:45.938676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 433/860] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:46:45 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:46:45.939910 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a70c0-3429-447f-aa08-fcd0f1cdfdf7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:45.952750 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f03a84-66ec-43bc-b6a5-7a8ebd4bb365 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:45.991492 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a08142-d129-49e3-b04b-60f7d403c35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:45.992010 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a08142-d129-49e3-b04b-60f7d403c35a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:45.992297 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a08142-d129-49e3-b04b-60f7d403c35a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:45.992627 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a08142-d129-49e3-b04b-60f7d403c35a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:45.996587 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:46.004857 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a08142-d129-49e3-b04b-60f7d403c35a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:46.006355 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a08142-d129-49e3-b04b-60f7d403c35a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:46.021897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-31a08142-d129-49e3-b04b-60f7d403c35a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:46.021897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:46.021897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:46.021897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:46.021897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:46.021897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:46.021897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:46.021897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:46.021897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:46.021897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:46.021897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:46.021897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:46.021897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:46.021897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:46.021897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:46.021897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:46.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:46.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:46.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:46.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:46.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:46.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:46.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:46.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:46.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:46.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:46.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:46.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:46.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:46.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:46.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:46.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:46.022866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:46.023804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:46.023804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:46.023804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:46.023804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 428/861] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:46.031665 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a08142-d129-49e3-b04b-60f7d403c35a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:46.058863 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:46.067717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a70c0-3429-447f-aa08-fcd0f1cdfdf7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:46.112236 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d4cbb-cc15-4393-9cd0-cc78bb1f310d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:46.113333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d4cbb-cc15-4393-9cd0-cc78bb1f310d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:46.113728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d4cbb-cc15-4393-9cd0-cc78bb1f310d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:46.114142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d4cbb-cc15-4393-9cd0-cc78bb1f310d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:46.128283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d4cbb-cc15-4393-9cd0-cc78bb1f310d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:46.131235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d4cbb-cc15-4393-9cd0-cc78bb1f310d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:46.150641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-694d4cbb-cc15-4393-9cd0-cc78bb1f310d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:46.150641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:46.150641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:46.150641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:46.150641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:46.150641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:46.150641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:46.150641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:46.150641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:46.150641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:46.150641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:46.150641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:46.150641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:46.150641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:46.150641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:46.150641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:46.151835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:46.151835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:46.151835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:46.151835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:46.151835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:46.151835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:46.151835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:46.151835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:46.151835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:46.151835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:46.151835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:46.151835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:46.151835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:46.151835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:46.151835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:46.151835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:46.151835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:46.153597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:46.153597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:46.153597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:46.159903 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 434/862] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:46.161378 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d4cbb-cc15-4393-9cd0-cc78bb1f310d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:46.167796 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a08142-d129-49e3-b04b-60f7d403c35a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:46.204246 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0a61d-a9fd-40e4-932d-20cfd4aceb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:46.204858 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0a61d-a9fd-40e4-932d-20cfd4aceb6f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:46.205141 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0a61d-a9fd-40e4-932d-20cfd4aceb6f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:46.205419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0a61d-a9fd-40e4-932d-20cfd4aceb6f None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:46.206164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0a61d-a9fd-40e4-932d-20cfd4aceb6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:46.206552 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0a61d-a9fd-40e4-932d-20cfd4aceb6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:46.208228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0a61d-a9fd-40e4-932d-20cfd4aceb6f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:46.220894 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 429/863] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:46.226513 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0a61d-a9fd-40e4-932d-20cfd4aceb6f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:46.287651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:46.293138 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d4cbb-cc15-4393-9cd0-cc78bb1f310d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:46.317189 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:46.335653 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25745e30-39c0-4955-b66d-af0b4e61fdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:46.336455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25745e30-39c0-4955-b66d-af0b4e61fdc8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:46.336886 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25745e30-39c0-4955-b66d-af0b4e61fdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:46.337273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25745e30-39c0-4955-b66d-af0b4e61fdc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:46.338232 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25745e30-39c0-4955-b66d-af0b4e61fdc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:46.338651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25745e30-39c0-4955-b66d-af0b4e61fdc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:46.340756 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25745e30-39c0-4955-b66d-af0b4e61fdc8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:46.349418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0a61d-a9fd-40e4-932d-20cfd4aceb6f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:46.358836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 435/864] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:46.359860 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25745e30-39c0-4955-b66d-af0b4e61fdc8 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:46.388551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1709727-cf96-41f7-947a-85ebc42524a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:46.389178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1709727-cf96-41f7-947a-85ebc42524a5 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:46.389510 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1709727-cf96-41f7-947a-85ebc42524a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:46.389731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1709727-cf96-41f7-947a-85ebc42524a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:46.396449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1709727-cf96-41f7-947a-85ebc42524a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:46.397936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1709727-cf96-41f7-947a-85ebc42524a5 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:46.409634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-c1709727-cf96-41f7-947a-85ebc42524a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:46.409634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:46.409634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:46.409634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:46.409634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:46.409634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:46.409634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:46.409634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:46.409634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:46.409634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:46.409634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:46.409634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:46.409634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:46.409634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:46.409634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:46.410695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:46.410695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:46.410695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:46.410695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:46.410695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:46.410695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:46.410695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:46.410695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:46.410695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:46.410695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:46.410695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:46.410695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:46.410695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:46.410695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:46.410695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:46.410695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:46.410695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:46.411603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:46.411603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:46.411603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:46.411603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:46.411603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:46.411603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:46.411603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:46.411603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:46.411603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 430/865] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:46.412220 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91b009-12d6-4019-80a3-265ce219e410 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:46.412220 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91b009-12d6-4019-80a3-265ce219e410 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:46.412373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91b009-12d6-4019-80a3-265ce219e410 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:46.413839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 431/866] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:46.416208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91b009-12d6-4019-80a3-265ce219e410 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:46.478710 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25745e30-39c0-4955-b66d-af0b4e61fdc8 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:46.516647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e313d-2b91-4098-ac11-bfee3709cb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:46.517891 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e313d-2b91-4098-ac11-bfee3709cb55 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:46.518181 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e313d-2b91-4098-ac11-bfee3709cb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:46.518463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e313d-2b91-4098-ac11-bfee3709cb55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:46.526294 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e313d-2b91-4098-ac11-bfee3709cb55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:46.527896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e313d-2b91-4098-ac11-bfee3709cb55 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:46.528096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91b009-12d6-4019-80a3-265ce219e410 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:46.539371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-9b9e313d-2b91-4098-ac11-bfee3709cb55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:46.539371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:46.539371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:46.539371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:46.539371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:46.539371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:46.539371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:46.539371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:46.539371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:46.539371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:46.539371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:46.539371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:46.539371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:46.539371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:46.539371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:46.540223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:46.540223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:46.540223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:46.540223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:46.540223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:46.540223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:46.540223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:46.540223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:46.540223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:46.540223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:46.540223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:46.540223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:46.540223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:46.540223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:46.540223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:46.540223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:46.540223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:46.541170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:46.541170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:46.541170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:46.541170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:46.541170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:46.541170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:46.541170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:46.541170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:46.541170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 436/867] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:46.541653 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ace13-3da6-466f-81c3-e8f41f2b3609 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:46.541653 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ace13-3da6-466f-81c3-e8f41f2b3609 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:46.542385 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ace13-3da6-466f-81c3-e8f41f2b3609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:46.562739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ef0ace13-3da6-466f-81c3-e8f41f2b3609 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:46.563928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ef0ace13-3da6-466f-81c3-e8f41f2b3609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:46.571866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17fedc-f4b3-421b-b7be-2a1717fdf927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:46.572470 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17fedc-f4b3-421b-b7be-2a1717fdf927 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:46.572751 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17fedc-f4b3-421b-b7be-2a1717fdf927 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:46.573027 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17fedc-f4b3-421b-b7be-2a1717fdf927 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:46.573721 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17fedc-f4b3-421b-b7be-2a1717fdf927 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:46.574057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17fedc-f4b3-421b-b7be-2a1717fdf927 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:46.576025 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17fedc-f4b3-421b-b7be-2a1717fdf927 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:46.578351 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:46.579581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:46.594382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 432/868] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:46.594382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17fedc-f4b3-421b-b7be-2a1717fdf927 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:46.640928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 437/869] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:46.641586 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ace13-3da6-466f-81c3-e8f41f2b3609 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:46.717566 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17fedc-f4b3-421b-b7be-2a1717fdf927 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:46.747990 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ace13-3da6-466f-81c3-e8f41f2b3609 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:46.764712 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1261eeb-9878-4ed7-b206-5edaf48be30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:46.765909 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1261eeb-9878-4ed7-b206-5edaf48be30e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:46.766767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1261eeb-9878-4ed7-b206-5edaf48be30e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:46.767259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1261eeb-9878-4ed7-b206-5edaf48be30e None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:46.768164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1261eeb-9878-4ed7-b206-5edaf48be30e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:46.768681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1261eeb-9878-4ed7-b206-5edaf48be30e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:46.771131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1261eeb-9878-4ed7-b206-5edaf48be30e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:46.793143 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 433/870] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:46.795553 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1261eeb-9878-4ed7-b206-5edaf48be30e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:46.798197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577cf0a-c2de-4dda-b5c4-59d6f01a8e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:46.799063 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577cf0a-c2de-4dda-b5c4-59d6f01a8e53 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:46.799347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577cf0a-c2de-4dda-b5c4-59d6f01a8e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:46.799622 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577cf0a-c2de-4dda-b5c4-59d6f01a8e53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:46.800351 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3577cf0a-c2de-4dda-b5c4-59d6f01a8e53 None admin] RBAC: Adding query filter params (role.id=d1bb55f25dff4694b8675e0ef5fc4ec7, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=d76a164ed7e14d9eb2db016b68a84d9a) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:46.800658 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3577cf0a-c2de-4dda-b5c4-59d6f01a8e53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:46.802128 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3577cf0a-c2de-4dda-b5c4-59d6f01a8e53 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:46.813884 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 438/871] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:46:46 2022] GET /identity/v3/role_assignments?user.id=d76a164ed7e14d9eb2db016b68a84d9a&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:46.881354 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:46.910949 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577cf0a-c2de-4dda-b5c4-59d6f01a8e53 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:46.930570 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:46.941309 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1261eeb-9878-4ed7-b206-5edaf48be30e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:46.988341 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ec9be-db15-4174-a032-8c096197d4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:46.989432 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ec9be-db15-4174-a032-8c096197d4cd None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:46.990263 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ec9be-db15-4174-a032-8c096197d4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:46.990849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ec9be-db15-4174-a032-8c096197d4cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:46.991892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ec9be-db15-4174-a032-8c096197d4cd None admin] RBAC: Adding query filter params (role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=d1047c7d18c1466d989cd895117cd807) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:46.992383 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ec9be-db15-4174-a032-8c096197d4cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:46.994206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ec9be-db15-4174-a032-8c096197d4cd None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:47.002549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 434/872] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:46:46 2022] GET /identity/v3/role_assignments?user.id=d1047c7d18c1466d989cd895117cd807&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:47.041714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577cf0a-c2de-4dda-b5c4-59d6f01a8e53 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:47.082213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb978be-3569-4a78-81ef-b2779dd094a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:47.082764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb978be-3569-4a78-81ef-b2779dd094a4 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:47.083045 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb978be-3569-4a78-81ef-b2779dd094a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:47.083326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb978be-3569-4a78-81ef-b2779dd094a4 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:47.084216 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb978be-3569-4a78-81ef-b2779dd094a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:47.085689 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb978be-3569-4a78-81ef-b2779dd094a4 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:47.164455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 439/873] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 14 11:46:46 2022] POST /identity/v3/services => generated 269 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:46:47.285724 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af1456-5a48-4177-9f4f-5518bac5258d None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:47.286064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af1456-5a48-4177-9f4f-5518bac5258d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:47.286337 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af1456-5a48-4177-9f4f-5518bac5258d None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:47.288046 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 435/874] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:47.294566 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42b8c3-7f14-4bf4-8ce4-b778969b55e0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:47.294860 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42b8c3-7f14-4bf4-8ce4-b778969b55e0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:47.295144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42b8c3-7f14-4bf4-8ce4-b778969b55e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:47.314834 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7f42b8c3-7f14-4bf4-8ce4-b778969b55e0 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:47.316108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7f42b8c3-7f14-4bf4-8ce4-b778969b55e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:47.388585 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:47.395390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 440/875] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:47 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:47.672534 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af1456-5a48-4177-9f4f-5518bac5258d None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:47.778398 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af1456-5a48-4177-9f4f-5518bac5258d None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:47.799105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:47.812994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656e88a-5d36-48df-978d-fabf60e1967c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:47.813606 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656e88a-5d36-48df-978d-fabf60e1967c None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:47.813909 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656e88a-5d36-48df-978d-fabf60e1967c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:47.814192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656e88a-5d36-48df-978d-fabf60e1967c None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:47.814860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656e88a-5d36-48df-978d-fabf60e1967c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:47.816225 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656e88a-5d36-48df-978d-fabf60e1967c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:47.902052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 436/876] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 14 11:46:47 2022] POST /identity/v3/services => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:46:49.170072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54a919-375f-4970-b9d6-1440cbf46110 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:49.171117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54a919-375f-4970-b9d6-1440cbf46110 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:49.171633 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54a919-375f-4970-b9d6-1440cbf46110 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:49.173471 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 441/877] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:49.179011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bd5e8-a8aa-4238-8e03-ff912025df45 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:49.179442 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bd5e8-a8aa-4238-8e03-ff912025df45 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:49.179623 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bd5e8-a8aa-4238-8e03-ff912025df45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:49.198722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f21bd5e8-a8aa-4238-8e03-ff912025df45 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:49.199950 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f21bd5e8-a8aa-4238-8e03-ff912025df45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:49.235297 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400a712-2946-4fc5-9d4f-b4d0f4cb8f87 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:49.235639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400a712-2946-4fc5-9d4f-b4d0f4cb8f87 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:49.236468 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400a712-2946-4fc5-9d4f-b4d0f4cb8f87 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:49.241150 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 442/878] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:46:49.242644 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9cd5f-5b3d-4df0-a0bf-e10e4b03d51b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:49.242929 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9cd5f-5b3d-4df0-a0bf-e10e4b03d51b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:49.243271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9cd5f-5b3d-4df0-a0bf-e10e4b03d51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:49.262986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1ef9cd5f-5b3d-4df0-a0bf-e10e4b03d51b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:49.264506 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1ef9cd5f-5b3d-4df0-a0bf-e10e4b03d51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:49.280187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 437/879] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:49.340076 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 443/880] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:49.405776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a832e7-97a2-4be1-977d-5a9840eb12fe None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:49.406101 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a832e7-97a2-4be1-977d-5a9840eb12fe None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:49.406384 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a832e7-97a2-4be1-977d-5a9840eb12fe None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:49.410013 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 438/881] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:49.416637 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca525b1b-2948-45e2-ae46-47c4db5b8d48 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:49.416932 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca525b1b-2948-45e2-ae46-47c4db5b8d48 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:49.417217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca525b1b-2948-45e2-ae46-47c4db5b8d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:49.435616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ca525b1b-2948-45e2-ae46-47c4db5b8d48 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:49.436724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ca525b1b-2948-45e2-ae46-47c4db5b8d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:49.505356 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 444/882] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:49.553007 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a832e7-97a2-4be1-977d-5a9840eb12fe None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:49.628969 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca525b1b-2948-45e2-ae46-47c4db5b8d48 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:49.656449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a832e7-97a2-4be1-977d-5a9840eb12fe None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:49.670274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:49.698985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a6b0f-316e-44c3-acb3-357a1f384784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:49.698985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a6b0f-316e-44c3-acb3-357a1f384784 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:49.698985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a6b0f-316e-44c3-acb3-357a1f384784 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:49.698985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a6b0f-316e-44c3-acb3-357a1f384784 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:49.699880 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:49.710576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a6b0f-316e-44c3-acb3-357a1f384784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:49.715490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-7e1a6b0f-316e-44c3-acb3-357a1f384784 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 11:46:49.715490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:49.715490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:49.715490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:49.715490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:49.715490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:49.715490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:49.715490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:49.715490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:49.715490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:49.715490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:49.715490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:49.715490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:49.715490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:49.715490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:49.715490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:49.716451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:49.716451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:49.716451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:49.716451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:49.716451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:49.716451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:49.716451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:49.716451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:49.716451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:49.716451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:49.716451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:49.716451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:49.716451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:49.716451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:49.716451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:49.716451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:49.717292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:49.717292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:49.717292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:49.717292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:49.717292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:49.717292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:49.717292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:49.717292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:49.717292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:49.717292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:49.717292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 11:46:49.717292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:49.717292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 439/883] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Sep 14 11:46:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:49.726246 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a6b0f-316e-44c3-acb3-357a1f384784 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:49.742637 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca525b1b-2948-45e2-ae46-47c4db5b8d48 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:49.779276 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed65ad-4d83-4cab-a16b-a7c9b7a5a4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:49.780230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed65ad-4d83-4cab-a16b-a7c9b7a5a4aa None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:49.780588 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed65ad-4d83-4cab-a16b-a7c9b7a5a4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:49.781131 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed65ad-4d83-4cab-a16b-a7c9b7a5a4aa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:49.781790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed65ad-4d83-4cab-a16b-a7c9b7a5a4aa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:49.802407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed65ad-4d83-4cab-a16b-a7c9b7a5a4aa None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:49.802407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-3aed65ad-4d83-4cab-a16b-a7c9b7a5a4aa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 11:46:49.802407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:49.802407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:46:49.802407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:46:49.802407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:46:49.802407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:46:49.802407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:46:49.802407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:49.802407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 11:46:49.802407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:49.802407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:49.802407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:49.802407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:49.802407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:49.803336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:49.803336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:49.803336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:49.803336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:49.803336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:49.803336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:49.803336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:49.803336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:49.803336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:49.803336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:46:49.803336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:46:49.803336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:46:49.803336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:46:49.803336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:49.803336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:49.803336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:49.803336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:49.804228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:49.804228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:49.804228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:49.804228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:49.804228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:49.804228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:49.804228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:49.804228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:49.804228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:49.804228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:49.804228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:49.804228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:49.804228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:46:49.804228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:49.804228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 11:46:49.804228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:49.805102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 445/884] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 14 11:46:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:49.805102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed65ad-4d83-4cab-a16b-a7c9b7a5a4aa None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:49.851210 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a6b0f-316e-44c3-acb3-357a1f384784 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:49.887941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad0db7-7035-420f-9632-085a6d8371a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:49.888554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad0db7-7035-420f-9632-085a6d8371a7 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:49.888841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad0db7-7035-420f-9632-085a6d8371a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:49.889121 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad0db7-7035-420f-9632-085a6d8371a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:49.889791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad0db7-7035-420f-9632-085a6d8371a7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:49.890149 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad0db7-7035-420f-9632-085a6d8371a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:49.891531 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad0db7-7035-420f-9632-085a6d8371a7 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:49.904555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 440/885] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Sep 14 11:46:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:49.904555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad0db7-7035-420f-9632-085a6d8371a7 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:49.918574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed65ad-4d83-4cab-a16b-a7c9b7a5a4aa None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:49.934503 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:49.943304 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:49.955271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3332dd7-a227-47a2-9da9-8dc7d859c3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:49.955910 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3332dd7-a227-47a2-9da9-8dc7d859c3e5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:49.956193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3332dd7-a227-47a2-9da9-8dc7d859c3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:49.956468 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3332dd7-a227-47a2-9da9-8dc7d859c3e5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:49.957060 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3332dd7-a227-47a2-9da9-8dc7d859c3e5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:49.958463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3332dd7-a227-47a2-9da9-8dc7d859c3e5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:49.967784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-e3332dd7-a227-47a2-9da9-8dc7d859c3e5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 11:46:49.967784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:49.967784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:46:49.967784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:46:49.967784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:46:49.967784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:46:49.967784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:46:49.967784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:49.967784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 11:46:49.967784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:49.967784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:49.967784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:49.967784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:49.967784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:49.967784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:49.967784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:49.968959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:49.968959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:49.968959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:49.968959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:49.968959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:49.968959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:49.968959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:49.968959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:46:49.968959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:46:49.968959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:46:49.968959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:46:49.968959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:49.968959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:49.968959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:49.968959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:49.968959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:49.968959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:49.970068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:49.970068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:49.970068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:49.970068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:49.970068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:49.970068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:49.970068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:49.970068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:49.970068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:49.970068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:49.970068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:46:49.970068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:49.970068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 11:46:49.970068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:49.970068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 446/886] 10.222.0.62 () {56 vars in 1132 bytes} [Wed Sep 14 11:46:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:49.971081 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3332dd7-a227-47a2-9da9-8dc7d859c3e5 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:50.015219 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad0db7-7035-420f-9632-085a6d8371a7 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:50.052902 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdeaa49-109c-4fff-b52f-a0622bb75cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:50.053512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdeaa49-109c-4fff-b52f-a0622bb75cca None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:50.054201 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdeaa49-109c-4fff-b52f-a0622bb75cca None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:50.054201 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdeaa49-109c-4fff-b52f-a0622bb75cca None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:50.054872 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdeaa49-109c-4fff-b52f-a0622bb75cca None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:50.055181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdeaa49-109c-4fff-b52f-a0622bb75cca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:50.056554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdeaa49-109c-4fff-b52f-a0622bb75cca None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:50.061900 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 441/887] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 14 11:46:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:50.062696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdeaa49-109c-4fff-b52f-a0622bb75cca None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:50.086681 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3332dd7-a227-47a2-9da9-8dc7d859c3e5 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:50.124015 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da80de0-76ba-46c4-9954-7a2d27026def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:50.124651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da80de0-76ba-46c4-9954-7a2d27026def None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:50.124934 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da80de0-76ba-46c4-9954-7a2d27026def None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:50.125208 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da80de0-76ba-46c4-9954-7a2d27026def None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:50.135776 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da80de0-76ba-46c4-9954-7a2d27026def None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:50.137239 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da80de0-76ba-46c4-9954-7a2d27026def None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:50.151705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-7da80de0-76ba-46c4-9954-7a2d27026def None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:50.151705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:50.151705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:50.151705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:50.151705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:50.151705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:50.151705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:50.151705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:50.151705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:50.151705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:50.151705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:50.151705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:50.151705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:50.151705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:50.151705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:50.151705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:50.152912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:50.152912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:50.152912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:50.152912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:50.152912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:50.152912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:50.152912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:50.152912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:50.152912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:50.152912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:50.152912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:50.152912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:50.152912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:50.152912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:50.152912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:50.152912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:50.152912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:50.154035 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:50.154035 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:50.154035 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:50.157327 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 447/888] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:50.158216 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da80de0-76ba-46c4-9954-7a2d27026def None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:50.158545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:50.167116 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdeaa49-109c-4fff-b52f-a0622bb75cca None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:50.198473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:50.202091 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63d7aa-9d46-4863-af09-9d69fba970c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:50.202705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63d7aa-9d46-4863-af09-9d69fba970c4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:50.202989 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63d7aa-9d46-4863-af09-9d69fba970c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:50.203267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63d7aa-9d46-4863-af09-9d69fba970c4 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:50.203899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63d7aa-9d46-4863-af09-9d69fba970c4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:50.204208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63d7aa-9d46-4863-af09-9d69fba970c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:50.205624 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63d7aa-9d46-4863-af09-9d69fba970c4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:50.210418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 442/889] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 14 11:46:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:50.213378 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f21cc4-d277-414f-b0ef-9e7ca4a5029b None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:50.213378 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f21cc4-d277-414f-b0ef-9e7ca4a5029b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:50.213378 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f21cc4-d277-414f-b0ef-9e7ca4a5029b None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:50.215255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 443/890] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:50.216283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f21cc4-d277-414f-b0ef-9e7ca4a5029b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:50.260726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da80de0-76ba-46c4-9954-7a2d27026def None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:50.294518 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829bfc8-4dab-4871-a6ae-8439759e25b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:50.295115 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829bfc8-4dab-4871-a6ae-8439759e25b5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:50.295396 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829bfc8-4dab-4871-a6ae-8439759e25b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:50.295669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829bfc8-4dab-4871-a6ae-8439759e25b5 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:50.296296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9829bfc8-4dab-4871-a6ae-8439759e25b5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:50.296602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9829bfc8-4dab-4871-a6ae-8439759e25b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:50.297995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9829bfc8-4dab-4871-a6ae-8439759e25b5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:50.302532 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 448/891] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 14 11:46:50 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:50.303302 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829bfc8-4dab-4871-a6ae-8439759e25b5 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:50.338369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f21cc4-d277-414f-b0ef-9e7ca4a5029b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:50.379467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2652f4-058c-4103-8cc3-cf77926bafb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:50.380095 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2652f4-058c-4103-8cc3-cf77926bafb2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:50.380394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2652f4-058c-4103-8cc3-cf77926bafb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:50.380671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2652f4-058c-4103-8cc3-cf77926bafb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:50.381360 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2652f4-058c-4103-8cc3-cf77926bafb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:50.381651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2652f4-058c-4103-8cc3-cf77926bafb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:50.383099 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2652f4-058c-4103-8cc3-cf77926bafb2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:50.400653 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 444/892] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:50.400653 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5e82a-b6b1-4f89-ba68-a7852f68ec74 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:50.400653 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5e82a-b6b1-4f89-ba68-a7852f68ec74 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:50.400653 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5e82a-b6b1-4f89-ba68-a7852f68ec74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:50.417789 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4ef5e82a-b6b1-4f89-ba68-a7852f68ec74 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:50.419585 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4ef5e82a-b6b1-4f89-ba68-a7852f68ec74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:50.424258 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829bfc8-4dab-4871-a6ae-8439759e25b5 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:50.425943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:50.431921 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:50.462973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a5e6b-b4cc-4eed-b689-436a10484f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:50.463602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a5e6b-b4cc-4eed-b689-436a10484f4b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:50.463796 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a5e6b-b4cc-4eed-b689-436a10484f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:50.464071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a5e6b-b4cc-4eed-b689-436a10484f4b None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:50.465236 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a5e6b-b4cc-4eed-b689-436a10484f4b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:50.465542 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a5e6b-b4cc-4eed-b689-436a10484f4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:50.467490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a5e6b-b4cc-4eed-b689-436a10484f4b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:50.474313 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 449/893] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 14 11:46:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:50.475278 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a5e6b-b4cc-4eed-b689-436a10484f4b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:50.503654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 445/894] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:50 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:50.504528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5e82a-b6b1-4f89-ba68-a7852f68ec74 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:50.580911 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a5e6b-b4cc-4eed-b689-436a10484f4b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:50.626046 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5e82a-b6b1-4f89-ba68-a7852f68ec74 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:50.626996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c57d40-b9b4-4142-8e6b-61da49d5f061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:50.627635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c57d40-b9b4-4142-8e6b-61da49d5f061 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:50.627957 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c57d40-b9b4-4142-8e6b-61da49d5f061 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:50.628232 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c57d40-b9b4-4142-8e6b-61da49d5f061 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:50.639485 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c57d40-b9b4-4142-8e6b-61da49d5f061 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:50.641218 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c57d40-b9b4-4142-8e6b-61da49d5f061 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:50.657665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-58c57d40-b9b4-4142-8e6b-61da49d5f061 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 11:46:50.657665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:50.657665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:50.657665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:50.657665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:50.657665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:50.657665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:50.657665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:50.657665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:50.657665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:50.657665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:50.657665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:50.657665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:50.657665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:50.657665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:50.657665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:50.658671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:50.658671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:50.658671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:50.658671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:50.658671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:50.658671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:50.658671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:50.658671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:50.658671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:50.658671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:50.658671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:50.658671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:50.658671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:50.658671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:50.658671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:50.658671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:50.658671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:50.658671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:50.659629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:50.659629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:50.659629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:50.659629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:50.659629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:50.659629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:50.659629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:50.659629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:50.659629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 11:46:50.659629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:50.659629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 450/895] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 14 11:46:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:50.666002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c57d40-b9b4-4142-8e6b-61da49d5f061 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:50.672798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ca284-faa3-4d04-a87d-f4d7fc24b6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:50.674071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ca284-faa3-4d04-a87d-f4d7fc24b6a1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:50.674545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ca284-faa3-4d04-a87d-f4d7fc24b6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:50.674999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ca284-faa3-4d04-a87d-f4d7fc24b6a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:50.675944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ca284-faa3-4d04-a87d-f4d7fc24b6a1 None admin] RBAC: Adding query filter params (interface=public, service_id=8f42d16e1cd846fea37a49b44d8ddb64, region_id=RegionOne) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:50.676432 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ca284-faa3-4d04-a87d-f4d7fc24b6a1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:50.678036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ca284-faa3-4d04-a87d-f4d7fc24b6a1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:50.686588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 446/896] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Sep 14 11:46:50 2022] GET /identity/v3/endpoints?service_id=8f42d16e1cd846fea37a49b44d8ddb64&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:50.698137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:50.702031 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ca284-faa3-4d04-a87d-f4d7fc24b6a1 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:50.719006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:50.797328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c57d40-b9b4-4142-8e6b-61da49d5f061 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:50.829077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ca284-faa3-4d04-a87d-f4d7fc24b6a1 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:50.841040 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca35a8-b0cb-437b-bed8-32c16a37b02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:50.841662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca35a8-b0cb-437b-bed8-32c16a37b02c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:50.841972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca35a8-b0cb-437b-bed8-32c16a37b02c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:50.842412 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca35a8-b0cb-437b-bed8-32c16a37b02c None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:50.843528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca35a8-b0cb-437b-bed8-32c16a37b02c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:50.843845 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca35a8-b0cb-437b-bed8-32c16a37b02c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:50.845701 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca35a8-b0cb-437b-bed8-32c16a37b02c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:50.863079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4c7b6-7761-4811-a384-07ceb8e2bb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:50.863689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4c7b6-7761-4811-a384-07ceb8e2bb7c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:50.863974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4c7b6-7761-4811-a384-07ceb8e2bb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:50.864253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4c7b6-7761-4811-a384-07ceb8e2bb7c None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:50.864863 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4c7b6-7761-4811-a384-07ceb8e2bb7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:50.866261 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4c7b6-7761-4811-a384-07ceb8e2bb7c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:50.871202 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 451/897] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Sep 14 11:46:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:50.871202 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca35a8-b0cb-437b-bed8-32c16a37b02c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:50.878067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 447/898] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Sep 14 11:46:50 2022] GET /identity/v3/services => generated 1268 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 11:46:50.878067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4c7b6-7761-4811-a384-07ceb8e2bb7c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:50.990281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:50.990603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:51.001892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4c7b6-7761-4811-a384-07ceb8e2bb7c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:51.008600 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca35a8-b0cb-437b-bed8-32c16a37b02c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:51.041968 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e7a2b-5032-4f48-aca2-223eb96cabef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:51.042892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e7a2b-5032-4f48-aca2-223eb96cabef None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:51.043323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e7a2b-5032-4f48-aca2-223eb96cabef None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:51.043690 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e7a2b-5032-4f48-aca2-223eb96cabef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:51.048386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c82df-792e-41d0-91c0-942902613381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:51.049296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c82df-792e-41d0-91c0-942902613381 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:51.049577 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c82df-792e-41d0-91c0-942902613381 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:51.049872 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c82df-792e-41d0-91c0-942902613381 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:51.050478 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c82df-792e-41d0-91c0-942902613381 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:51.053288 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c82df-792e-41d0-91c0-942902613381 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:51.066303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-bd5c82df-792e-41d0-91c0-942902613381 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 11:46:51.066303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:51.066303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:46:51.066303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:46:51.066303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:46:51.066303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:46:51.066303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:46:51.066303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:51.066303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 11:46:51.066303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:51.066303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:51.066303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:51.066303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:51.066303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:51.066303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:51.066303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:51.066303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:51.067346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:51.067346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:51.067346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:51.067346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:51.067346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:51.067346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:51.067346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:46:51.067346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:46:51.067346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:46:51.067346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:46:51.067346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:51.067346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:51.067346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:51.067346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:51.067346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:51.067346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:51.068188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:51.068188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:51.068188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:51.068188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:51.068188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:51.068188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:51.068188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:51.068188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:51.068188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:51.068188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:51.068188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:46:51.068188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:51.068188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 11:46:51.068188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:51.068188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 452/899] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:46:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:51.076626 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c82df-792e-41d0-91c0-942902613381 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:51.128736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e7a2b-5032-4f48-aca2-223eb96cabef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:51.130587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e7a2b-5032-4f48-aca2-223eb96cabef None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:51.217567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:51.225430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c82df-792e-41d0-91c0-942902613381 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:51.242692 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 448/900] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:46:50 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:46:51.248029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e7a2b-5032-4f48-aca2-223eb96cabef None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:51.270649 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49cb7d1-5b85-4de8-a7d1-333c4c83cd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:51.271969 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cb7d1-5b85-4de8-a7d1-333c4c83cd0c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:51.272406 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cb7d1-5b85-4de8-a7d1-333c4c83cd0c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:51.272836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cb7d1-5b85-4de8-a7d1-333c4c83cd0c None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:51.273618 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49cb7d1-5b85-4de8-a7d1-333c4c83cd0c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:51.274110 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49cb7d1-5b85-4de8-a7d1-333c4c83cd0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:51.276497 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49cb7d1-5b85-4de8-a7d1-333c4c83cd0c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:51.289072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 453/901] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:46:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:46:51.290297 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49cb7d1-5b85-4de8-a7d1-333c4c83cd0c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:51.307446 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:51.391438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e7a2b-5032-4f48-aca2-223eb96cabef None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:51.397900 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49cb7d1-5b85-4de8-a7d1-333c4c83cd0c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:51.434216 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49853d48-94db-4fc4-886a-e20cbc544c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:51.435010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49853d48-94db-4fc4-886a-e20cbc544c5a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:51.435385 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49853d48-94db-4fc4-886a-e20cbc544c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:51.435812 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49853d48-94db-4fc4-886a-e20cbc544c5a None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:51.436568 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49853d48-94db-4fc4-886a-e20cbc544c5a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:51.436756 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507fbcf-028b-4d81-95d0-1aa3db1ead2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:51.437072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49853d48-94db-4fc4-886a-e20cbc544c5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:51.437705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507fbcf-028b-4d81-95d0-1aa3db1ead2a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:51.438189 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507fbcf-028b-4d81-95d0-1aa3db1ead2a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:51.438621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507fbcf-028b-4d81-95d0-1aa3db1ead2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:51.438782 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49853d48-94db-4fc4-886a-e20cbc544c5a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:51.448306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 454/902] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:46:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:51.449472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49853d48-94db-4fc4-886a-e20cbc544c5a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:51.458912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507fbcf-028b-4d81-95d0-1aa3db1ead2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:51.461089 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507fbcf-028b-4d81-95d0-1aa3db1ead2a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:51.481713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-0507fbcf-028b-4d81-95d0-1aa3db1ead2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:51.481713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:51.481713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:51.481713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:51.481713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:51.481713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:51.481713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:51.481713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:51.481713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:51.481713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:51.481713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:51.481713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:51.481713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:51.481713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:51.481713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:51.481713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:51.483011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:51.483011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:51.483011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:51.483011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:51.483011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:51.483011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:51.483011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:51.483011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:51.483011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:51.483011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:51.483011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:51.483011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:51.483011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:51.483011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:51.483011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:51.483011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:51.483011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:51.484221 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:51.484221 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:51.484221 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:51.486383 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 449/903] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:51.494896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507fbcf-028b-4d81-95d0-1aa3db1ead2a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:51.530207 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:51.538454 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:51.570486 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49853d48-94db-4fc4-886a-e20cbc544c5a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:51.615309 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a609b9-4c92-4d5a-ba2e-7d492f2956b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:51.615765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507fbcf-028b-4d81-95d0-1aa3db1ead2a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:51.616454 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a609b9-4c92-4d5a-ba2e-7d492f2956b6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:51.616907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a609b9-4c92-4d5a-ba2e-7d492f2956b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:51.617549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a609b9-4c92-4d5a-ba2e-7d492f2956b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:51.619279 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a609b9-4c92-4d5a-ba2e-7d492f2956b6 None admin] RBAC: Adding query filter params (interface=public, service_id=f9ed0d7b0e6a4f889579a1d5ba2d595d, region_id=RegionOne) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:51.619770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a609b9-4c92-4d5a-ba2e-7d492f2956b6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:51.621348 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a609b9-4c92-4d5a-ba2e-7d492f2956b6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:51.628507 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 455/904] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Sep 14 11:46:51 2022] GET /identity/v3/endpoints?service_id=f9ed0d7b0e6a4f889579a1d5ba2d595d&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:51.633965 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a609b9-4c92-4d5a-ba2e-7d492f2956b6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:51.660166 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed366f-ac50-4ad8-bc2e-61bac7252745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:51.661086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed366f-ac50-4ad8-bc2e-61bac7252745 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:51.661406 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed366f-ac50-4ad8-bc2e-61bac7252745 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:51.661688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed366f-ac50-4ad8-bc2e-61bac7252745 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:51.662630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed366f-ac50-4ad8-bc2e-61bac7252745 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:51.663320 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed366f-ac50-4ad8-bc2e-61bac7252745 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:51.664701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed366f-ac50-4ad8-bc2e-61bac7252745 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:51.689381 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 450/905] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:51.690442 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed366f-ac50-4ad8-bc2e-61bac7252745 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:51.754880 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a609b9-4c92-4d5a-ba2e-7d492f2956b6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:51.790600 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f6880-03bb-48b8-85db-f89a0452f8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:51.791723 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f6880-03bb-48b8-85db-f89a0452f8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:51.792003 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f6880-03bb-48b8-85db-f89a0452f8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:51.792272 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f6880-03bb-48b8-85db-f89a0452f8a3 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:51.792913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f6880-03bb-48b8-85db-f89a0452f8a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:51.794755 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f6880-03bb-48b8-85db-f89a0452f8a3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:51.802817 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:51.804356 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 456/906] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Sep 14 11:46:51 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 11:46:51.840513 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed366f-ac50-4ad8-bc2e-61bac7252745 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:51.881637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e3b09-0a55-49b1-aa22-b00e0d27cd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:51.882479 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e3b09-0a55-49b1-aa22-b00e0d27cd7c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:51.882849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e3b09-0a55-49b1-aa22-b00e0d27cd7c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:51.883206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e3b09-0a55-49b1-aa22-b00e0d27cd7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:51.891909 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e3b09-0a55-49b1-aa22-b00e0d27cd7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:51.893555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e3b09-0a55-49b1-aa22-b00e0d27cd7c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:51.907594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-3b4e3b09-0a55-49b1-aa22-b00e0d27cd7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:51.907594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:51.907594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:51.907594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:51.907594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:51.907594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:51.907594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:51.907594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:51.907594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:51.907594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:51.907594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:51.907594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:51.907594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:51.907594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:51.907594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:51.908807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:51.908807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:51.908807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:51.908807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:51.908807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:51.908807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:51.908807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:51.908807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:51.908807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:51.908807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:51.908807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:51.908807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:51.908807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:51.908807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:51.908807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:51.908807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:51.908807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:51.909979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:51.909979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:51.909979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:51.909979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:51.909979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:51.909979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:51.909979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:51.909979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:51.916882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f6880-03bb-48b8-85db-f89a0452f8a3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:51.917323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 451/907] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:51.920982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:52.026366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f6880-03bb-48b8-85db-f89a0452f8a3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:52.060809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c687d-d57c-40be-ba37-29aaf0dbb8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:52.061429 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c687d-d57c-40be-ba37-29aaf0dbb8a8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:52.061716 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c687d-d57c-40be-ba37-29aaf0dbb8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:52.062029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c687d-d57c-40be-ba37-29aaf0dbb8a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:52.062726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c687d-d57c-40be-ba37-29aaf0dbb8a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:52.063036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c687d-d57c-40be-ba37-29aaf0dbb8a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:52.064464 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c687d-d57c-40be-ba37-29aaf0dbb8a8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:52.078847 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 457/908] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:52.084973 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e3b09-0a55-49b1-aa22-b00e0d27cd7c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:52.190351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e3b09-0a55-49b1-aa22-b00e0d27cd7c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:52.226333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa1975-ba5f-42a5-b5e4-17c92c994002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:52.226970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa1975-ba5f-42a5-b5e4-17c92c994002 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:52.227259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa1975-ba5f-42a5-b5e4-17c92c994002 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:52.227598 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa1975-ba5f-42a5-b5e4-17c92c994002 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:52.228338 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa1975-ba5f-42a5-b5e4-17c92c994002 None admin] RBAC: Adding query filter params (role.id=09d8286638fb49d8a4499662066fda1a, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=d76a164ed7e14d9eb2db016b68a84d9a) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:52.228652 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa1975-ba5f-42a5-b5e4-17c92c994002 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:52.230166 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa1975-ba5f-42a5-b5e4-17c92c994002 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:52.236317 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 452/909] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:46:52 2022] GET /identity/v3/role_assignments?user.id=d76a164ed7e14d9eb2db016b68a84d9a&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=09d8286638fb49d8a4499662066fda1a => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:52.515395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931d6d2-cb19-49cb-9c2b-a32b28de90f5 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:52.515942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931d6d2-cb19-49cb-9c2b-a32b28de90f5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:52.515942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931d6d2-cb19-49cb-9c2b-a32b28de90f5 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:52.517653 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 458/910] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:52.523043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973f227-d1ea-4afb-a5ef-6a0d24cb4360 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:52.523333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973f227-d1ea-4afb-a5ef-6a0d24cb4360 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:52.523610 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973f227-d1ea-4afb-a5ef-6a0d24cb4360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:52.542366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c973f227-d1ea-4afb-a5ef-6a0d24cb4360 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:52.544134 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c973f227-d1ea-4afb-a5ef-6a0d24cb4360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:52.547119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:52.620533 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 453/911] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:52.888957 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931d6d2-cb19-49cb-9c2b-a32b28de90f5 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:52.955143 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:53.000569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931d6d2-cb19-49cb-9c2b-a32b28de90f5 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:53.034835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104bb58-2853-40eb-90d9-929d1f89ceae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:53.035447 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104bb58-2853-40eb-90d9-929d1f89ceae None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:53.035732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104bb58-2853-40eb-90d9-929d1f89ceae None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:53.036009 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104bb58-2853-40eb-90d9-929d1f89ceae None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:53.036678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6104bb58-2853-40eb-90d9-929d1f89ceae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:53.038063 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6104bb58-2853-40eb-90d9-929d1f89ceae None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:53.090586 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 459/912] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 14 11:46:52 2022] POST /identity/v3/services => generated 241 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:46:53.209768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea0909-46fe-44fa-8fab-5f6e5712bb7a None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:53.210136 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea0909-46fe-44fa-8fab-5f6e5712bb7a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:53.210365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea0909-46fe-44fa-8fab-5f6e5712bb7a None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:53.211998 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 454/913] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:53.218048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8f155-f08f-4a61-9260-3f5d111b3a1c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:53.218690 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8f155-f08f-4a61-9260-3f5d111b3a1c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:53.219181 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8f155-f08f-4a61-9260-3f5d111b3a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:53.239589 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a6f8f155-f08f-4a61-9260-3f5d111b3a1c None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:53.241261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a6f8f155-f08f-4a61-9260-3f5d111b3a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:53.318046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 460/914] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:53.588892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea0909-46fe-44fa-8fab-5f6e5712bb7a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:53.702569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea0909-46fe-44fa-8fab-5f6e5712bb7a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:53.762393 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a79a9-7019-4687-81d9-40f94e7d6016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:53.763037 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a79a9-7019-4687-81d9-40f94e7d6016 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:53.763321 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a79a9-7019-4687-81d9-40f94e7d6016 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:53.763596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a79a9-7019-4687-81d9-40f94e7d6016 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:53.764245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a79a9-7019-4687-81d9-40f94e7d6016 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:53.765646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a79a9-7019-4687-81d9-40f94e7d6016 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:53.774671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-a94a79a9-7019-4687-81d9-40f94e7d6016 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 11:46:53.774671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:53.774671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:46:53.774671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:46:53.774671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:46:53.774671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:46:53.774671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:46:53.774671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:53.774671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 11:46:53.774671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:53.774671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:53.774671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:53.774671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:53.774671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:53.774671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:53.774671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:53.775772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:53.775772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:53.775772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:53.775772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:53.775772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:53.775772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:53.775772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:53.775772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:46:53.775772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:46:53.775772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:46:53.775772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:46:53.775772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:53.775772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:53.775772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:53.775772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:53.775772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:53.775772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:53.776828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:53.776828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:53.776828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:53.776828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:53.776828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:53.776828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:53.776828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:53.776828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:53.776828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:53.776828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:53.776828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:46:53.776828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:53.776828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 11:46:53.776828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:53.776828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 455/915] 10.222.0.62 () {56 vars in 1132 bytes} [Wed Sep 14 11:46:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:53.788680 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8f155-f08f-4a61-9260-3f5d111b3a1c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:53.845284 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:53.897822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8f155-f08f-4a61-9260-3f5d111b3a1c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:53.934661 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad61a3-b133-4a22-a878-4c0f96c985d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:53.935581 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad61a3-b133-4a22-a878-4c0f96c985d3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:53.936107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad61a3-b133-4a22-a878-4c0f96c985d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:53.936575 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad61a3-b133-4a22-a878-4c0f96c985d3 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:53.937400 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad61a3-b133-4a22-a878-4c0f96c985d3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:53.937922 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad61a3-b133-4a22-a878-4c0f96c985d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:53.939480 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad61a3-b133-4a22-a878-4c0f96c985d3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:53.949905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 461/916] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 14 11:46:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:53.952816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a79a9-7019-4687-81d9-40f94e7d6016 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:53.960669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:54.119805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a79a9-7019-4687-81d9-40f94e7d6016 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:54.167966 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fd645-9e05-4e6d-a9c4-550b3194d416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:54.168674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fd645-9e05-4e6d-a9c4-550b3194d416 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:54.168964 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fd645-9e05-4e6d-a9c4-550b3194d416 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:54.169246 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fd645-9e05-4e6d-a9c4-550b3194d416 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:54.169926 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fd645-9e05-4e6d-a9c4-550b3194d416 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:54.170239 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fd645-9e05-4e6d-a9c4-550b3194d416 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:54.171644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fd645-9e05-4e6d-a9c4-550b3194d416 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:54.187610 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 456/917] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 14 11:46:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:54.189931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad61a3-b133-4a22-a878-4c0f96c985d3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:54.195537 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fc657-a9da-4ec3-a9d4-232ff8728d4a None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:54.196038 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fc657-a9da-4ec3-a9d4-232ff8728d4a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:54.196465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fc657-a9da-4ec3-a9d4-232ff8728d4a None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:54.198201 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 457/918] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:54.206160 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe0a55-fd19-476a-a231-7595c53cf875 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:54.206654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe0a55-fd19-476a-a231-7595c53cf875 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:54.207184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe0a55-fd19-476a-a231-7595c53cf875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:54.234786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-2cbe0a55-fd19-476a-a231-7595c53cf875 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:54.237995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2cbe0a55-fd19-476a-a231-7595c53cf875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:54.319451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad61a3-b133-4a22-a878-4c0f96c985d3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:54.338151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:54.346654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:54.358042 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 458/919] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 11:46:54.361840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836a8c2-eb6c-4299-9b81-86f64683b3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:54.362723 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836a8c2-eb6c-4299-9b81-86f64683b3e0 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:54.363222 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836a8c2-eb6c-4299-9b81-86f64683b3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:54.363731 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836a8c2-eb6c-4299-9b81-86f64683b3e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:54.364607 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836a8c2-eb6c-4299-9b81-86f64683b3e0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:54.366273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836a8c2-eb6c-4299-9b81-86f64683b3e0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:54.490047 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 462/920] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 11:46:54 2022] POST /identity/v3/endpoints => generated 347 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 11:46:54.508686 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c9ba5-8442-4db4-9b71-0ca83e0dfe89 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:54.509027 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c9ba5-8442-4db4-9b71-0ca83e0dfe89 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:54.509276 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c9ba5-8442-4db4-9b71-0ca83e0dfe89 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:54.510930 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 459/921] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:54.516585 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888276a-6027-424b-8cf4-e1fcea95b8c9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:54.516906 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888276a-6027-424b-8cf4-e1fcea95b8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:54.517140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888276a-6027-424b-8cf4-e1fcea95b8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:54.537322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-c888276a-6027-424b-8cf4-e1fcea95b8c9 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:54.538638 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c888276a-6027-424b-8cf4-e1fcea95b8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:54.620162 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 463/922] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:54.635984 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c9ba5-8442-4db4-9b71-0ca83e0dfe89 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:54.751642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c9ba5-8442-4db4-9b71-0ca83e0dfe89 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:54.787654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f97d55-ad7d-469a-849d-8205ed19a144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:54.788284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f97d55-ad7d-469a-849d-8205ed19a144 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:54.788563 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f97d55-ad7d-469a-849d-8205ed19a144 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:54.788831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f97d55-ad7d-469a-849d-8205ed19a144 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:54.789418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f97d55-ad7d-469a-849d-8205ed19a144 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:54.790798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f97d55-ad7d-469a-849d-8205ed19a144 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:54.800481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-07f97d55-ad7d-469a-849d-8205ed19a144 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 11:46:54.800481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:54.800481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:46:54.800481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:46:54.800481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:46:54.800481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:46:54.800481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:46:54.800481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:54.800481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 11:46:54.800481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:54.800481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:54.800481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:54.800481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:54.800481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:54.800481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:54.800481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:54.800481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:54.801474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:54.801474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:54.801474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:54.801474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:54.801474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:54.801474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:54.801474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:46:54.801474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:46:54.801474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:46:54.801474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:46:54.801474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:54.801474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:54.801474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:54.801474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:54.801474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:54.801474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:54.802349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:54.802349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:54.802349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:54.802349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:54.802349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:54.802349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:54.802349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:54.802349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:54.802349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:54.802349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:54.802349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:46:54.802349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:54.802349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 11:46:54.802349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:54.802349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 460/923] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:46:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:54.810619 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c888276a-6027-424b-8cf4-e1fcea95b8c9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:54.893771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f97d55-ad7d-469a-849d-8205ed19a144 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:54.933079 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:54.944064 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c888276a-6027-424b-8cf4-e1fcea95b8c9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:54.982388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:54.991235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b3f22-b7ca-40fc-b51d-b87cdb05bd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:54.992347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b3f22-b7ca-40fc-b51d-b87cdb05bd04 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:54.992786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b3f22-b7ca-40fc-b51d-b87cdb05bd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:54.993486 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b3f22-b7ca-40fc-b51d-b87cdb05bd04 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:54.994617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b3f22-b7ca-40fc-b51d-b87cdb05bd04 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:54.995374 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b3f22-b7ca-40fc-b51d-b87cdb05bd04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:54.997932 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b3f22-b7ca-40fc-b51d-b87cdb05bd04 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:55.012721 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 464/924] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:46:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:55.012721 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b3f22-b7ca-40fc-b51d-b87cdb05bd04 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:55.018232 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f97d55-ad7d-469a-849d-8205ed19a144 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:55.061642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68540bf3-1a05-4745-9974-384e605b7b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:55.062464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68540bf3-1a05-4745-9974-384e605b7b15 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:55.062856 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68540bf3-1a05-4745-9974-384e605b7b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:55.063784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68540bf3-1a05-4745-9974-384e605b7b15 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:55.078002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68540bf3-1a05-4745-9974-384e605b7b15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:46:55.085901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-68540bf3-1a05-4745-9974-384e605b7b15 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 11:46:55.085901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:55.085901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:55.085901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:55.085901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:55.085901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:55.085901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:55.085901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:55.085901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:55.085901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:55.085901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:55.085901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:55.085901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:46:55.085901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:46:55.085901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:46:55.085901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:46:55.086954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:46:55.086954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:46:55.086954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:46:55.086954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:46:55.086954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:55.086954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:55.086954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:55.086954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:55.086954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:55.086954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:55.086954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:55.086954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:55.086954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:55.086954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:55.086954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:55.086954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:55.087802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:55.087802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:55.087802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:55.087802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:55.087802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:46:55.087802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:46:55.087802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:46:55.087802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:46:55.087802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:46:55.087802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:46:55.087802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 11:46:55.087802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:55.090091 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 461/925] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Sep 14 11:46:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:55.093314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68540bf3-1a05-4745-9974-384e605b7b15 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:55.136634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b3f22-b7ca-40fc-b51d-b87cdb05bd04 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:55.172577 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fb956-9685-4cae-a7bd-107abad878f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:55.173925 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fb956-9685-4cae-a7bd-107abad878f4 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:55.174129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fb956-9685-4cae-a7bd-107abad878f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:55.174508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fb956-9685-4cae-a7bd-107abad878f4 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:55.175238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fb956-9685-4cae-a7bd-107abad878f4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:55.175647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fb956-9685-4cae-a7bd-107abad878f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:55.177115 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fb956-9685-4cae-a7bd-107abad878f4 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:55.182889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 465/926] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 14 11:46:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:55.189381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fb956-9685-4cae-a7bd-107abad878f4 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:55.210476 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68540bf3-1a05-4745-9974-384e605b7b15 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:55.220992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:55.234984 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:55.251576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec4c74-8344-404b-9b6f-c0716d1b64da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:55.252240 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec4c74-8344-404b-9b6f-c0716d1b64da None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:55.252800 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec4c74-8344-404b-9b6f-c0716d1b64da None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:55.253137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec4c74-8344-404b-9b6f-c0716d1b64da None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:55.254006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec4c74-8344-404b-9b6f-c0716d1b64da None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:55.254325 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec4c74-8344-404b-9b6f-c0716d1b64da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:55.256119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec4c74-8344-404b-9b6f-c0716d1b64da None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:55.268522 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 462/927] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Sep 14 11:46:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:46:55.274219 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec4c74-8344-404b-9b6f-c0716d1b64da None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:55.309378 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fb956-9685-4cae-a7bd-107abad878f4 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:55.350302 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca71db-60c2-4b45-98b1-a2e4de1a277a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:55.351049 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca71db-60c2-4b45-98b1-a2e4de1a277a None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:55.351398 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca71db-60c2-4b45-98b1-a2e4de1a277a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:55.351767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca71db-60c2-4b45-98b1-a2e4de1a277a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:55.352549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca71db-60c2-4b45-98b1-a2e4de1a277a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:55.354208 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca71db-60c2-4b45-98b1-a2e4de1a277a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:55.407589 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec4c74-8344-404b-9b6f-c0716d1b64da None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:55.446212 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a6742-3e50-4058-a457-6d30c67c5600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:55.447005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a6742-3e50-4058-a457-6d30c67c5600 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:55.447297 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a6742-3e50-4058-a457-6d30c67c5600 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:55.447641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a6742-3e50-4058-a457-6d30c67c5600 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:55.458719 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a6742-3e50-4058-a457-6d30c67c5600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:55.460179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a6742-3e50-4058-a457-6d30c67c5600 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:55.472735 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 466/928] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 11:46:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 11:46:55.474494 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf195e-7a0e-467a-90f6-01bf181af02d None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:55.474988 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf195e-7a0e-467a-90f6-01bf181af02d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:55.475469 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf195e-7a0e-467a-90f6-01bf181af02d None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:55.476468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-303a6742-3e50-4058-a457-6d30c67c5600 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:55.476468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:55.476468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:55.476468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:55.476468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:55.476468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:55.476468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:55.476468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:55.476468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:55.476468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:55.476468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:55.476468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:55.476468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:55.476468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:55.476468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:55.476468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:55.477402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:55.477402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:55.477402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:55.477402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:55.477402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:55.477402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:55.477402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:55.477402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:55.477402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:55.477402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:55.477402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:55.477402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:55.477402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:55.477402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:55.477402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:55.477402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:55.477402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:55.478963 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:55.478963 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:55.478963 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:55.478963 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 463/929] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:55.479960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 467/930] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:55.491020 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a6742-3e50-4058-a457-6d30c67c5600 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:55.491442 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4090905-2089-4b22-b54b-79aedd9e2a92 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:55.491860 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4090905-2089-4b22-b54b-79aedd9e2a92 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:55.492138 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4090905-2089-4b22-b54b-79aedd9e2a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:55.518750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-b4090905-2089-4b22-b54b-79aedd9e2a92 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:55.521088 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b4090905-2089-4b22-b54b-79aedd9e2a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:55.540403 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:55.627334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 468/931] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:55.645056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a6742-3e50-4058-a457-6d30c67c5600 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:55.682176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c009f6-7c51-4c2d-b45b-6c29492850f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:55.682841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c009f6-7c51-4c2d-b45b-6c29492850f8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:55.683154 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c009f6-7c51-4c2d-b45b-6c29492850f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:55.683440 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c009f6-7c51-4c2d-b45b-6c29492850f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:55.684090 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c009f6-7c51-4c2d-b45b-6c29492850f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:55.684397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c009f6-7c51-4c2d-b45b-6c29492850f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:55.685775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c009f6-7c51-4c2d-b45b-6c29492850f8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:55.698970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 464/932] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:55.704417 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4090905-2089-4b22-b54b-79aedd9e2a92 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:55.713530 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:55.814616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4090905-2089-4b22-b54b-79aedd9e2a92 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:55.862282 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f61a2-af86-4f5f-b709-7eac5c8437a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:55.862915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f61a2-af86-4f5f-b709-7eac5c8437a9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:55.863203 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f61a2-af86-4f5f-b709-7eac5c8437a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:55.863479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f61a2-af86-4f5f-b709-7eac5c8437a9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:55.870891 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f61a2-af86-4f5f-b709-7eac5c8437a9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:55.872339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f61a2-af86-4f5f-b709-7eac5c8437a9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:55.884095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-f76f61a2-af86-4f5f-b709-7eac5c8437a9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 11:46:55.884095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:55.884095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:55.884095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:55.884095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:55.884095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:55.884095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:55.884095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:55.884095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:55.884095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:55.884095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:55.884095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:55.884095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:46:55.884095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:46:55.884095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:46:55.884095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:46:55.885092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:55.885092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:55.885092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:46:55.885092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:55.885092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:46:55.885092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:46:55.885092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:55.885092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:55.885092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:55.885092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:55.885092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:55.885092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:55.885092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:55.885092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:55.885092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:55.885092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:55.885092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:55.885092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:55.887186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:55.887186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:55.887186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:46:55.887186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:46:55.887186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:46:55.887186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:46:55.887186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:46:55.887186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:46:55.887186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 11:46:55.887186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:55.887186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 469/933] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 14 11:46:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:55.891606 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c009f6-7c51-4c2d-b45b-6c29492850f8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:55.893666 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f61a2-af86-4f5f-b709-7eac5c8437a9 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:55.968037 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:55.998086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:56.004406 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c009f6-7c51-4c2d-b45b-6c29492850f8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:56.018193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f61a2-af86-4f5f-b709-7eac5c8437a9 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:56.056387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661dea40-81c1-442a-b138-b544fb85d18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:56.057006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dea40-81c1-442a-b138-b544fb85d18b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:56.057293 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dea40-81c1-442a-b138-b544fb85d18b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:56.057572 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dea40-81c1-442a-b138-b544fb85d18b None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:56.057987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456695f9-9081-4a95-961f-4fdbeb6c72e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:56.058406 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661dea40-81c1-442a-b138-b544fb85d18b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:56.058656 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456695f9-9081-4a95-961f-4fdbeb6c72e0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:56.058760 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661dea40-81c1-442a-b138-b544fb85d18b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:56.058944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456695f9-9081-4a95-961f-4fdbeb6c72e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:56.059221 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456695f9-9081-4a95-961f-4fdbeb6c72e0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:56.059808 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456695f9-9081-4a95-961f-4fdbeb6c72e0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:56.060171 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661dea40-81c1-442a-b138-b544fb85d18b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:56.061169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456695f9-9081-4a95-961f-4fdbeb6c72e0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:56.071523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-456695f9-9081-4a95-961f-4fdbeb6c72e0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 11:46:56.071523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:56.071523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:46:56.071523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:46:56.071523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:46:56.071523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:46:56.071523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:46:56.071523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:56.071523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 11:46:56.071523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:56.071523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:56.071523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:56.071523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:56.071523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:56.071523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:56.071523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:56.071523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:56.072517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:56.072517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:56.072517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:56.072517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:56.072517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:56.072517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:56.072517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:46:56.072517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:46:56.072517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:46:56.072517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:46:56.072517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:56.072517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:56.072517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:56.072517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:56.072517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:56.072517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:56.073655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:56.073655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:56.073655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:56.073655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:56.073655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:56.073655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:56.073655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:56.073655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:56.073655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:56.073655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:56.073655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:46:56.073655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:56.073655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 11:46:56.073655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:56.073655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 470/934] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 14 11:46:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:56.078934 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456695f9-9081-4a95-961f-4fdbeb6c72e0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:56.085020 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 465/935] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Sep 14 11:46:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:56.086872 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661dea40-81c1-442a-b138-b544fb85d18b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:56.200070 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456695f9-9081-4a95-961f-4fdbeb6c72e0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:56.215230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661dea40-81c1-442a-b138-b544fb85d18b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:56.230451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:56.236694 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b11585-298c-461c-bf5a-251c5c89390a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:56.237428 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b11585-298c-461c-bf5a-251c5c89390a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:56.237816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b11585-298c-461c-bf5a-251c5c89390a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:56.238216 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b11585-298c-461c-bf5a-251c5c89390a None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:56.238959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b11585-298c-461c-bf5a-251c5c89390a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:56.239371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b11585-298c-461c-bf5a-251c5c89390a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:56.240866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b11585-298c-461c-bf5a-251c5c89390a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:56.247028 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 471/936] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 14 11:46:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:56.281675 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b11585-298c-461c-bf5a-251c5c89390a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:56.281823 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc43e67-1bad-4c5a-8ee2-850b7f779c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:56.281823 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc43e67-1bad-4c5a-8ee2-850b7f779c94 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:56.281823 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc43e67-1bad-4c5a-8ee2-850b7f779c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:56.281823 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc43e67-1bad-4c5a-8ee2-850b7f779c94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:56.282226 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc43e67-1bad-4c5a-8ee2-850b7f779c94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:56.282226 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc43e67-1bad-4c5a-8ee2-850b7f779c94 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:56.294974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-8dc43e67-1bad-4c5a-8ee2-850b7f779c94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:56.294974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:56.294974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:56.294974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:56.294974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:56.294974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:56.294974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:56.294974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:56.294974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:56.294974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:56.294974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:56.294974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:56.294974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:46:56.294974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:46:56.294974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:46:56.294974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:46:56.296183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:56.296183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:56.296183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:56.296183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:56.296183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:56.296183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:56.296183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:56.296183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:56.296183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:56.296183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:56.296183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:56.296183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:56.296183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:56.296183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:56.296183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:56.296183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:56.296183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:46:56.297333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:46:56.297333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:46:56.297333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:56.299930 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 466/937] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:56.304798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc43e67-1bad-4c5a-8ee2-850b7f779c94 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:56.315577 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:56.373209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b11585-298c-461c-bf5a-251c5c89390a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:56.411077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af667208-621f-4cb8-b9d6-592ff81b14f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:56.411813 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af667208-621f-4cb8-b9d6-592ff81b14f1 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:56.412180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af667208-621f-4cb8-b9d6-592ff81b14f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:56.412506 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af667208-621f-4cb8-b9d6-592ff81b14f1 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:56.413208 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af667208-621f-4cb8-b9d6-592ff81b14f1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:56.413580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af667208-621f-4cb8-b9d6-592ff81b14f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:56.415172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af667208-621f-4cb8-b9d6-592ff81b14f1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:56.425814 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 472/938] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 14 11:46:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:56.426902 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af667208-621f-4cb8-b9d6-592ff81b14f1 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:56.438439 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc43e67-1bad-4c5a-8ee2-850b7f779c94 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:56.476894 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb284a-a945-416d-b266-8f9fd4a24346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:56.477558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb284a-a945-416d-b266-8f9fd4a24346 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:56.477851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb284a-a945-416d-b266-8f9fd4a24346 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:56.478200 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb284a-a945-416d-b266-8f9fd4a24346 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:56.478861 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb284a-a945-416d-b266-8f9fd4a24346 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:56.479168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb284a-a945-416d-b266-8f9fd4a24346 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:56.480703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb284a-a945-416d-b266-8f9fd4a24346 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:56.500238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 467/939] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:46:56.501798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb284a-a945-416d-b266-8f9fd4a24346 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:56.506915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:56.555842 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af667208-621f-4cb8-b9d6-592ff81b14f1 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:56.581113 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:56.592817 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e627dc3-c600-4d49-961d-8fa09a762df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:56.593427 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e627dc3-c600-4d49-961d-8fa09a762df7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:56.593738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e627dc3-c600-4d49-961d-8fa09a762df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:56.598123 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e627dc3-c600-4d49-961d-8fa09a762df7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:56.598123 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e627dc3-c600-4d49-961d-8fa09a762df7 None admin] RBAC: Adding query filter params (interface=public, service_id=bdc188a3ec984d5da35a7d7a047e0abf, region_id=RegionOne) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:56.598123 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e627dc3-c600-4d49-961d-8fa09a762df7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:56.598123 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e627dc3-c600-4d49-961d-8fa09a762df7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:56.606228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 473/940] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Sep 14 11:46:56 2022] GET /identity/v3/endpoints?service_id=bdc188a3ec984d5da35a7d7a047e0abf&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:56.608596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e627dc3-c600-4d49-961d-8fa09a762df7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:56.628184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb284a-a945-416d-b266-8f9fd4a24346 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:56.677649 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b2ce1-5144-40e5-bca8-77e3771a6db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:56.679109 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b2ce1-5144-40e5-bca8-77e3771a6db9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:56.679649 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b2ce1-5144-40e5-bca8-77e3771a6db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:56.680394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b2ce1-5144-40e5-bca8-77e3771a6db9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:56.689740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b2ce1-5144-40e5-bca8-77e3771a6db9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:56.692012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b2ce1-5144-40e5-bca8-77e3771a6db9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:56.708773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-348b2ce1-5144-40e5-bca8-77e3771a6db9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:56.708773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:56.708773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:56.708773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:56.708773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:56.708773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:56.708773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:56.708773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:56.708773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:56.708773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:56.708773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:56.708773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:56.708773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:56.708773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:56.708773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:56.709761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:56.709761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:56.709761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:56.709761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:56.709761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:56.709761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:56.709761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:56.709761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:56.709761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:56.709761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:56.709761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:56.709761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:56.709761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:56.709761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:56.709761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:56.709761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:56.709761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:56.710718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:56.710718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:56.710718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:56.710718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:56.710718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:56.710718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:56.710718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:56.710718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:56.715413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 468/941] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:56.716257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b2ce1-5144-40e5-bca8-77e3771a6db9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:56.724318 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e627dc3-c600-4d49-961d-8fa09a762df7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:56.754190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:56.769688 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af240f-4797-4148-bb41-dcc4cf53cce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:56.771159 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af240f-4797-4148-bb41-dcc4cf53cce7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:56.772172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af240f-4797-4148-bb41-dcc4cf53cce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:56.772766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af240f-4797-4148-bb41-dcc4cf53cce7 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:56.773669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af240f-4797-4148-bb41-dcc4cf53cce7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:56.776115 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af240f-4797-4148-bb41-dcc4cf53cce7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:56.785886 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 474/942] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Sep 14 11:46:56 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 11:46:56.841506 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b2ce1-5144-40e5-bca8-77e3771a6db9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:56.872634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:56.882508 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653d13e-dc8b-42a6-899f-9d051d47ef28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:56.883060 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653d13e-dc8b-42a6-899f-9d051d47ef28 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:56.883347 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653d13e-dc8b-42a6-899f-9d051d47ef28 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:56.883658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653d13e-dc8b-42a6-899f-9d051d47ef28 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:56.884388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653d13e-dc8b-42a6-899f-9d051d47ef28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:56.884704 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653d13e-dc8b-42a6-899f-9d051d47ef28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:56.886781 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653d13e-dc8b-42a6-899f-9d051d47ef28 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:56.894026 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d91760-952e-4786-a2c6-581a254c635e None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:56.894563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d91760-952e-4786-a2c6-581a254c635e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:56.894563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d91760-952e-4786-a2c6-581a254c635e None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:56.900437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 475/943] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:46:56.901343 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 469/944] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:46:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:56.903095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7384e9-c69f-4325-a4f5-9499983714b7 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:56.903539 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7384e9-c69f-4325-a4f5-9499983714b7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:56.903820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7384e9-c69f-4325-a4f5-9499983714b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:56.907946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653d13e-dc8b-42a6-899f-9d051d47ef28 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:56.924903 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-dc7384e9-c69f-4325-a4f5-9499983714b7 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:56.926760 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-dc7384e9-c69f-4325-a4f5-9499983714b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:57.019268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 476/945] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:57.046958 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653d13e-dc8b-42a6-899f-9d051d47ef28 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:57.107189 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8c1e0-8539-42bd-99bc-e1c1d90191d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:57.108321 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8c1e0-8539-42bd-99bc-e1c1d90191d4 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:57.108825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8c1e0-8539-42bd-99bc-e1c1d90191d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:57.109334 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8c1e0-8539-42bd-99bc-e1c1d90191d4 None admin] PATH_INFO: `/v3/projects/449a5fac1acd478abea544446c08ecd9/users/d76a164ed7e14d9eb2db016b68a84d9a/roles/09d8286638fb49d8a4499662066fda1a` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:57.113056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8c1e0-8539-42bd-99bc-e1c1d90191d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=449a5fac1acd478abea544446c08ecd9, user_id=d76a164ed7e14d9eb2db016b68a84d9a, role_id=09d8286638fb49d8a4499662066fda1a)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:57.115801 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8c1e0-8539-42bd-99bc-e1c1d90191d4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:57.192086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 470/946] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:46:56 2022] PUT /identity/v3/projects/449a5fac1acd478abea544446c08ecd9/users/d76a164ed7e14d9eb2db016b68a84d9a/roles/09d8286638fb49d8a4499662066fda1a => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:46:57.303648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7384e9-c69f-4325-a4f5-9499983714b7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:57.410814 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7384e9-c69f-4325-a4f5-9499983714b7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:57.428569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:57.448759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7f2e7-f52b-4068-a912-ab6dd0979210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:57.448759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7f2e7-f52b-4068-a912-ab6dd0979210 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:57.448759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7f2e7-f52b-4068-a912-ab6dd0979210 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:57.448759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7f2e7-f52b-4068-a912-ab6dd0979210 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:57.449205 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd7f2e7-f52b-4068-a912-ab6dd0979210 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:57.449986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd7f2e7-f52b-4068-a912-ab6dd0979210 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:57.458993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-8dd7f2e7-f52b-4068-a912-ab6dd0979210 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 11:46:57.458993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:57.458993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:46:57.458993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:46:57.458993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:46:57.458993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:46:57.458993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:46:57.458993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:57.458993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 11:46:57.458993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:57.458993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:57.458993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:57.458993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:57.458993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:57.458993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:57.458993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:57.458993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:57.459986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:57.459986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:57.459986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:57.459986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:57.459986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:57.459986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:57.459986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:46:57.459986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:46:57.459986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:46:57.459986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:46:57.459986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:57.459986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:57.459986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:57.459986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:57.459986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:57.459986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:57.460836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:57.460836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:57.460836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:57.460836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:57.460836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:57.460836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:57.460836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:57.460836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:57.460836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:57.460836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:57.460836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:46:57.460836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:57.460836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 11:46:57.460836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:57.464733 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 477/947] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:57.466635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8c1e0-8539-42bd-99bc-e1c1d90191d4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:57.516157 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:57.572078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8c1e0-8539-42bd-99bc-e1c1d90191d4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:57.608686 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2905e15-04cc-47e0-bd6d-4e4a471db6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:57.609303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2905e15-04cc-47e0-bd6d-4e4a471db6dc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:57.609583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2905e15-04cc-47e0-bd6d-4e4a471db6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:57.609874 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2905e15-04cc-47e0-bd6d-4e4a471db6dc None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:57.610523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2905e15-04cc-47e0-bd6d-4e4a471db6dc None admin] RBAC: Adding query filter params (name=compute) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:57.610826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2905e15-04cc-47e0-bd6d-4e4a471db6dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:57.612199 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2905e15-04cc-47e0-bd6d-4e4a471db6dc None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:57.622079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 471/948] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 14 11:46:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:57.624237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7f2e7-f52b-4068-a912-ab6dd0979210 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:57.731434 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7f2e7-f52b-4068-a912-ab6dd0979210 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:57.766415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddeb59e-ff86-4390-aa7a-33abfaf65354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:57.767406 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddeb59e-ff86-4390-aa7a-33abfaf65354 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:57.767901 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddeb59e-ff86-4390-aa7a-33abfaf65354 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:57.768379 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddeb59e-ff86-4390-aa7a-33abfaf65354 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:57.769208 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddeb59e-ff86-4390-aa7a-33abfaf65354 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:57.769744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddeb59e-ff86-4390-aa7a-33abfaf65354 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:57.771333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddeb59e-ff86-4390-aa7a-33abfaf65354 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:57.777608 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 478/949] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 14 11:46:57 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:57.921446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3293a-5bf4-4258-a6ee-e82888798f04 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:57.921987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3293a-5bf4-4258-a6ee-e82888798f04 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:57.922099 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3293a-5bf4-4258-a6ee-e82888798f04 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:57.923905 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 472/950] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:57.929656 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa6d25-cd73-43e1-8f57-7a30f46aa1f4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:57.930187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa6d25-cd73-43e1-8f57-7a30f46aa1f4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:57.930266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa6d25-cd73-43e1-8f57-7a30f46aa1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:57.949983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e1fa6d25-cd73-43e1-8f57-7a30f46aa1f4 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:57.952309 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e1fa6d25-cd73-43e1-8f57-7a30f46aa1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:57.996256 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:58.024225 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 479/951] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:58.302419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3293a-5bf4-4258-a6ee-e82888798f04 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:58.405783 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3293a-5bf4-4258-a6ee-e82888798f04 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:58.416751 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:58.440269 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310588d-bc91-4a15-a946-0dc98740af98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:58.440918 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310588d-bc91-4a15-a946-0dc98740af98 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:58.441203 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310588d-bc91-4a15-a946-0dc98740af98 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:58.441476 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310588d-bc91-4a15-a946-0dc98740af98 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:58.442157 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310588d-bc91-4a15-a946-0dc98740af98 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:58.443638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310588d-bc91-4a15-a946-0dc98740af98 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:58.446354 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: WARNING keystone.server.flask.common [None req-3310588d-bc91-4a15-a946-0dc98740af98 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 14 11:46:58.488173 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.sql.core [None req-3310588d-bc91-4a15-a946-0dc98740af98 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 14 11:46:58.488173 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 14 11:46:58.488173 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [parameters: {'id': '3f05deab48724bb8b9b987d54bf5022a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 14 11:46:58.488173 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81581) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 14 11:46:58.497029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-3310588d-bc91-4a15-a946-0dc98740af98 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3f05deab48724bb8b9b987d54bf5022a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3f05deab48724bb8b9b987d54bf5022a. Sep 14 11:46:58.497029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:58.497029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 11:46:58.497029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 11:46:58.497029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 11:46:58.497029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 11:46:58.497029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 11:46:58.497029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 11:46:58.497029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 11:46:58.497029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application conn.query(q) Sep 14 11:46:58.497029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 11:46:58.497029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 11:46:58.497029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 11:46:58.500408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result.read() Sep 14 11:46:58.500408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 11:46:58.500408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 11:46:58.500408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 11:46:58.500408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 11:46:58.500408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 11:46:58.500408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 11:46:58.500408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 11:46:58.500408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 11:46:58.500408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 14 11:46:58.500408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:58.500408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 14 11:46:58.500408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:58.500408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:58.500408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 14 11:46:58.500408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 14 11:46:58.500408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 14 11:46:58.500408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 14 11:46:58.501368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 11:46:58.501368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application next(self.gen) Sep 14 11:46:58.501368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 14 11:46:58.501368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application yield resource Sep 14 11:46:58.501368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 11:46:58.501368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application next(self.gen) Sep 14 11:46:58.501368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 14 11:46:58.501368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.session.rollback() Sep 14 11:46:58.501368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 14 11:46:58.501368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.force_reraise() Sep 14 11:46:58.501368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 14 11:46:58.501368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise self.value Sep 14 11:46:58.501368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 14 11:46:58.501368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 14 11:46:58.501368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 14 11:46:58.501368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application session.commit() Sep 14 11:46:58.501368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 14 11:46:58.502326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 14 11:46:58.502326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 14 11:46:58.502326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._prepare_impl() Sep 14 11:46:58.502326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 14 11:46:58.502326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.session.flush() Sep 14 11:46:58.502326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 14 11:46:58.502326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._flush(objects) Sep 14 11:46:58.502326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 14 11:46:58.502326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 14 11:46:58.502326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 14 11:46:58.502326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application compat.raise_( Sep 14 11:46:58.502326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 11:46:58.502326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception Sep 14 11:46:58.502326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 14 11:46:58.502326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application flush_context.execute() Sep 14 11:46:58.502326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 14 11:46:58.502326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rec.execute(self) Sep 14 11:46:58.503237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 14 11:46:58.503237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 14 11:46:58.503237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 14 11:46:58.503237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 14 11:46:58.503237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 14 11:46:58.503237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 14 11:46:58.503237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 14 11:46:58.503237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 14 11:46:58.503237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 14 11:46:58.503237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 14 11:46:58.503237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 14 11:46:58.503237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 14 11:46:58.503237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 14 11:46:58.503237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 14 11:46:58.503237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 14 11:46:58.505730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 14 11:46:58.505730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 11:46:58.505730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception Sep 14 11:46:58.505730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 11:46:58.505730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 11:46:58.505730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 11:46:58.505730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 11:46:58.505730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 11:46:58.505730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 11:46:58.505730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 11:46:58.505730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application conn.query(q) Sep 14 11:46:58.505730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 11:46:58.505730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 11:46:58.505730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 11:46:58.505730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result.read() Sep 14 11:46:58.505730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 11:46:58.505730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 11:46:58.507945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 11:46:58.507945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 11:46:58.507945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 11:46:58.507945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 11:46:58.507945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 11:46:58.507945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 11:46:58.507945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 14 11:46:58.507945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 14 11:46:58.507945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [parameters: {'id': '3f05deab48724bb8b9b987d54bf5022a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 14 11:46:58.507945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 14 11:46:58.507945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:58.507945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:58.507945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:58.507945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:58.507945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 14 11:46:58.509937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 14 11:46:58.509937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 14 11:46:58.509937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 14 11:46:58.509937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 14 11:46:58.509937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:58.509937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:58.509937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:58.509937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:58.509937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:58.509937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:58.509937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:58.509937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:58.509937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:58.509937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:58.509937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:58.509937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:58.509937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:58.512991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:58.512991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 14 11:46:58.512991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 14 11:46:58.512991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:58.512991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:58.512991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 14 11:46:58.512991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 14 11:46:58.512991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3f05deab48724bb8b9b987d54bf5022a. Sep 14 11:46:58.512991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:46:58.512991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 473/952] 10.222.0.62 () {60 vars in 1140 bytes} [Wed Sep 14 11:46:58 2022] POST /identity/v3/projects => generated 262 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:58.514315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa6d25-cd73-43e1-8f57-7a30f46aa1f4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:58.615671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa6d25-cd73-43e1-8f57-7a30f46aa1f4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:58.651186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75028763-3043-4b50-a35b-70f05eb26773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:58.652684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75028763-3043-4b50-a35b-70f05eb26773 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:58.653183 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75028763-3043-4b50-a35b-70f05eb26773 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:58.653665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75028763-3043-4b50-a35b-70f05eb26773 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:58.660800 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75028763-3043-4b50-a35b-70f05eb26773 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:58.662486 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75028763-3043-4b50-a35b-70f05eb26773 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:58.674818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-75028763-3043-4b50-a35b-70f05eb26773 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:58.674818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:58.674818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:58.674818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:58.674818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:58.674818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:58.674818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:58.674818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:58.674818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:58.674818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:58.674818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:58.674818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:58.674818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:46:58.674818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:46:58.674818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:46:58.675726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:46:58.675726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:58.675726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:58.675726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:58.675726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:58.675726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:58.675726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:58.675726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:58.675726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:58.675726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:58.675726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:58.675726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:58.675726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:58.675726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:58.675726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:58.675726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:58.675726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:58.676677 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:46:58.676677 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:46:58.676677 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:46:58.676677 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:46:58.676677 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:46:58.676677 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:46:58.676677 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:46:58.676677 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:58.679171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 480/953] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:46:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:46:58.685435 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310588d-bc91-4a15-a946-0dc98740af98 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:58.795901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310588d-bc91-4a15-a946-0dc98740af98 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:58.844165 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b132c8b-893e-4778-b858-9b08e65149f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:58.844986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b132c8b-893e-4778-b858-9b08e65149f8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:58.845419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b132c8b-893e-4778-b858-9b08e65149f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:58.845887 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b132c8b-893e-4778-b858-9b08e65149f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:58.846718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b132c8b-893e-4778-b858-9b08e65149f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:58.847183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b132c8b-893e-4778-b858-9b08e65149f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:58.848977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b132c8b-893e-4778-b858-9b08e65149f8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:58.870329 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 474/954] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 14 11:46:58 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:59.145578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9320f-b231-4e47-ae08-e165c35269c1 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:59.146090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9320f-b231-4e47-ae08-e165c35269c1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:59.146323 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9320f-b231-4e47-ae08-e165c35269c1 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:59.148027 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 481/955] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:59.154109 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2a4af-e19f-4899-8b90-ecfe0d2c7dc9 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:59.154937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2a4af-e19f-4899-8b90-ecfe0d2c7dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:59.155419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2a4af-e19f-4899-8b90-ecfe0d2c7dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:59.175027 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-49d2a4af-e19f-4899-8b90-ecfe0d2c7dc9 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:59.176895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-49d2a4af-e19f-4899-8b90-ecfe0d2c7dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:59.199144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:59.249598 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 475/956] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:59.496185 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d28990-8378-44b7-871f-160d88c09109 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:59.496534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d28990-8378-44b7-871f-160d88c09109 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:59.496658 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d28990-8378-44b7-871f-160d88c09109 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:59.498277 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 482/957] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:46:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:46:59.503517 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93e081-630b-4919-9bea-bcbe4d056a54 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:59.503817 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93e081-630b-4919-9bea-bcbe4d056a54 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:59.504099 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93e081-630b-4919-9bea-bcbe4d056a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:59.523518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-eb93e081-630b-4919-9bea-bcbe4d056a54 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:46:59.524727 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-eb93e081-630b-4919-9bea-bcbe4d056a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:46:59.527127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d28990-8378-44b7-871f-160d88c09109 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:59.565175 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:59.613884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 476/958] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:46:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:46:59.642510 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d28990-8378-44b7-871f-160d88c09109 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:59.680201 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a97a53-23a0-4c2c-b738-f769ad910359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:59.680840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a97a53-23a0-4c2c-b738-f769ad910359 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:59.681121 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a97a53-23a0-4c2c-b738-f769ad910359 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:59.681403 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a97a53-23a0-4c2c-b738-f769ad910359 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:59.682028 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a97a53-23a0-4c2c-b738-f769ad910359 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:59.684025 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a97a53-23a0-4c2c-b738-f769ad910359 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:59.693572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-43a97a53-23a0-4c2c-b738-f769ad910359 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 11:46:59.693572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:59.693572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:46:59.693572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:46:59.693572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:46:59.693572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:46:59.693572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:46:59.693572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:59.693572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 11:46:59.693572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:59.693572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:46:59.693572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:59.693572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:46:59.693572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:46:59.693572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:46:59.693572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:46:59.693572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:46:59.694678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:46:59.694678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:46:59.694678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:46:59.694678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:46:59.694678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:46:59.694678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:46:59.694678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:46:59.694678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:46:59.694678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:46:59.694678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:46:59.694678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:46:59.694678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:46:59.694678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:46:59.694678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:46:59.694678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:46:59.694678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:46:59.695960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:46:59.695960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:46:59.695960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:46:59.695960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:46:59.695960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:46:59.695960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:46:59.695960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:46:59.695960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:46:59.695960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:46:59.695960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:46:59.695960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:46:59.695960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:46:59.695960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 11:46:59.695960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:46:59.695960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 483/959] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 14 11:46:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:46:59.703239 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93e081-630b-4919-9bea-bcbe4d056a54 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:59.790733 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:59.806961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93e081-630b-4919-9bea-bcbe4d056a54 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:46:59.855403 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb9966-c562-4040-bf5d-403e50a0fb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:46:59.856471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb9966-c562-4040-bf5d-403e50a0fb8f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:46:59.856959 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb9966-c562-4040-bf5d-403e50a0fb8f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:46:59.857465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb9966-c562-4040-bf5d-403e50a0fb8f None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:46:59.858339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb9966-c562-4040-bf5d-403e50a0fb8f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:46:59.858867 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb9966-c562-4040-bf5d-403e50a0fb8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:46:59.860450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb9966-c562-4040-bf5d-403e50a0fb8f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:46:59.866091 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 477/960] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 14 11:46:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:46:59.875000 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a97a53-23a0-4c2c-b738-f769ad910359 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:59.910662 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb9966-c562-4040-bf5d-403e50a0fb8f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:46:59.993411 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:46:59.998134 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a97a53-23a0-4c2c-b738-f769ad910359 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:00.035081 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ec029-86d2-4de8-b98a-38a2c29b796c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:00.035689 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ec029-86d2-4de8-b98a-38a2c29b796c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:00.036024 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ec029-86d2-4de8-b98a-38a2c29b796c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:00.036300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ec029-86d2-4de8-b98a-38a2c29b796c None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:00.037013 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ec029-86d2-4de8-b98a-38a2c29b796c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:00.037322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ec029-86d2-4de8-b98a-38a2c29b796c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:00.038794 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ec029-86d2-4de8-b98a-38a2c29b796c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:00.049395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 484/961] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 14 11:46:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:00.049652 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb9966-c562-4040-bf5d-403e50a0fb8f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:00.050658 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ec029-86d2-4de8-b98a-38a2c29b796c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:00.093743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058ee3c-7c06-49a8-bf48-4cf903af203d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:00.094423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058ee3c-7c06-49a8-bf48-4cf903af203d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:00.094739 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058ee3c-7c06-49a8-bf48-4cf903af203d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:00.094993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058ee3c-7c06-49a8-bf48-4cf903af203d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:00.106219 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058ee3c-7c06-49a8-bf48-4cf903af203d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:47:00.111036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-5058ee3c-7c06-49a8-bf48-4cf903af203d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 11:47:00.111036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:00.111036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:00.111036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:00.111036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:00.111036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:00.111036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:00.111036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:00.111036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:00.111036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:00.111036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:00.111036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:00.111036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:47:00.111036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:47:00.111036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:47:00.111036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:47:00.111987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:47:00.111987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:47:00.111987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:47:00.111987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:47:00.111987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:00.111987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:00.111987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:00.111987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:00.111987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:00.111987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:00.111987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:00.111987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:00.111987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:00.111987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:00.111987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:00.111987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:00.112873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:00.112873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:00.112873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:00.112873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:00.112873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:47:00.112873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:47:00.112873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:47:00.112873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:47:00.112873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:47:00.112873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:47:00.112873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 11:47:00.112873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:00.112873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 478/962] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Sep 14 11:46:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:47:00.113588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cff5d-3141-4fb9-af23-f85fc5198d84 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:00.113588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cff5d-3141-4fb9-af23-f85fc5198d84 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:00.114266 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cff5d-3141-4fb9-af23-f85fc5198d84 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:00.115821 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 479/963] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:00.120419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cff5d-3141-4fb9-af23-f85fc5198d84 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:00.132257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:00.165276 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ec029-86d2-4de8-b98a-38a2c29b796c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:00.202136 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ee169-db46-42d2-8eac-2a01e844a62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:00.202757 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ee169-db46-42d2-8eac-2a01e844a62f None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:00.203196 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ee169-db46-42d2-8eac-2a01e844a62f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:00.203477 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ee169-db46-42d2-8eac-2a01e844a62f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:00.204145 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ee169-db46-42d2-8eac-2a01e844a62f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:00.205503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ee169-db46-42d2-8eac-2a01e844a62f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:00.228274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cff5d-3141-4fb9-af23-f85fc5198d84 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:00.262974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a0c7b-62a7-4f17-973d-3ec0ebf0dc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:00.263573 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a0c7b-62a7-4f17-973d-3ec0ebf0dc2e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:00.263893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a0c7b-62a7-4f17-973d-3ec0ebf0dc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:00.264426 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a0c7b-62a7-4f17-973d-3ec0ebf0dc2e None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:00.265085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a0c7b-62a7-4f17-973d-3ec0ebf0dc2e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:00.265391 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a0c7b-62a7-4f17-973d-3ec0ebf0dc2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:00.267030 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a0c7b-62a7-4f17-973d-3ec0ebf0dc2e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:00.272044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 485/964] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 11:47:00 2022] POST /identity/v3/endpoints => generated 349 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:47:00.273222 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645eedb1-1a7e-4189-a47b-7dc3155cf4e5 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:00.273509 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645eedb1-1a7e-4189-a47b-7dc3155cf4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:00.273787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645eedb1-1a7e-4189-a47b-7dc3155cf4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:00.289896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 480/965] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Sep 14 11:47:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:00.291194 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a0c7b-62a7-4f17-973d-3ec0ebf0dc2e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:00.300217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-645eedb1-1a7e-4189-a47b-7dc3155cf4e5 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:00.301375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-645eedb1-1a7e-4189-a47b-7dc3155cf4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:00.319763 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:00.378377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:00.400042 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 486/966] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:00.425895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a0c7b-62a7-4f17-973d-3ec0ebf0dc2e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:00.469386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32d725-18fe-49df-99d2-62248aa04532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:00.470052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32d725-18fe-49df-99d2-62248aa04532 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:00.470331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32d725-18fe-49df-99d2-62248aa04532 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:00.470605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32d725-18fe-49df-99d2-62248aa04532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:00.481597 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32d725-18fe-49df-99d2-62248aa04532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:00.483081 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32d725-18fe-49df-99d2-62248aa04532 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:00.497054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2b32d725-18fe-49df-99d2-62248aa04532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:47:00.497054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:00.497054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:00.497054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:00.497054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:00.497054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:00.497054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:00.497054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:00.497054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:00.497054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:00.497054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:00.497054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:00.497054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:47:00.497054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:47:00.497054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:47:00.497054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:47:00.498073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:00.498073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:00.498073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:00.498073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:00.498073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:00.498073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:00.498073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:00.498073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:00.498073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:00.498073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:00.498073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:00.498073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:00.498073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:00.498073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:00.498073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:00.498073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:00.498073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:47:00.498997 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:47:00.498997 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:47:00.498997 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:00.498997 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 481/967] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:47:00.503957 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645eedb1-1a7e-4189-a47b-7dc3155cf4e5 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:00.607254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645eedb1-1a7e-4189-a47b-7dc3155cf4e5 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:00.642617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbb4a3-ee39-4ac3-b758-d950d39f47bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:00.643225 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbb4a3-ee39-4ac3-b758-d950d39f47bd None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:00.643504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbb4a3-ee39-4ac3-b758-d950d39f47bd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:00.643775 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbb4a3-ee39-4ac3-b758-d950d39f47bd None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:00.644409 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbb4a3-ee39-4ac3-b758-d950d39f47bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:00.644713 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbb4a3-ee39-4ac3-b758-d950d39f47bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:00.646095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbb4a3-ee39-4ac3-b758-d950d39f47bd None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:00.659004 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 487/968] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:00.665329 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32d725-18fe-49df-99d2-62248aa04532 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:00.684382 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbb4a3-ee39-4ac3-b758-d950d39f47bd None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:00.688802 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:00.777068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32d725-18fe-49df-99d2-62248aa04532 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:00.796708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbb4a3-ee39-4ac3-b758-d950d39f47bd None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:00.802331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:00.825126 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6816c92-2482-4089-aa7e-95ddd1fd352b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:00.832312 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6816c92-2482-4089-aa7e-95ddd1fd352b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:00.833330 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6816c92-2482-4089-aa7e-95ddd1fd352b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:00.834004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6816c92-2482-4089-aa7e-95ddd1fd352b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:00.843761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c8ea1-b9ec-4605-8dce-da39f61957ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:00.844590 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c8ea1-b9ec-4605-8dce-da39f61957ea None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:00.844934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6816c92-2482-4089-aa7e-95ddd1fd352b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:00.845214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c8ea1-b9ec-4605-8dce-da39f61957ea None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:00.845717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c8ea1-b9ec-4605-8dce-da39f61957ea None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:00.846609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c8ea1-b9ec-4605-8dce-da39f61957ea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:00.846742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6816c92-2482-4089-aa7e-95ddd1fd352b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:00.848578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c8ea1-b9ec-4605-8dce-da39f61957ea None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:00.859805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-c6816c92-2482-4089-aa7e-95ddd1fd352b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 11:47:00.859805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:00.859805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:00.859805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:00.859805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:00.859805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:00.859805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:00.859805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:00.859805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:00.859805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:00.859805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:00.859805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:00.859805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:47:00.859805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:47:00.859805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:47:00.859805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:47:00.860871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:00.860871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:00.860871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:47:00.860871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:47:00.860871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:47:00.860871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:47:00.860871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:00.860871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:00.860871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:00.860871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:00.860871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:00.860871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:00.860871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:00.860871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:00.860871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:00.860871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:00.860871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:00.860871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:00.861938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:00.861938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:00.861938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:47:00.861938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:47:00.861938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:47:00.861938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:47:00.861938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:47:00.861938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:47:00.861938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 11:47:00.861938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:00.865653 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 482/969] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 14 11:47:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:00.866521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6816c92-2482-4089-aa7e-95ddd1fd352b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:00.926257 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 488/970] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 14 11:47:00 2022] POST /identity/v3/services => generated 242 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 11:47:00.995374 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6816c92-2482-4089-aa7e-95ddd1fd352b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:01.047257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2cd59-a8e4-45d1-8db2-4a60455cf506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:01.047890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2cd59-a8e4-45d1-8db2-4a60455cf506 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:01.048174 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2cd59-a8e4-45d1-8db2-4a60455cf506 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:01.048716 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2cd59-a8e4-45d1-8db2-4a60455cf506 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:01.049449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee2cd59-a8e4-45d1-8db2-4a60455cf506 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:01.049759 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee2cd59-a8e4-45d1-8db2-4a60455cf506 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:01.051256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee2cd59-a8e4-45d1-8db2-4a60455cf506 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:01.072176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 483/971] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Sep 14 11:47:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:01.078696 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c8ea1-b9ec-4605-8dce-da39f61957ea None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:01.130548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadef31-2597-411b-a54a-ed5c833e584d None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:01.130754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadef31-2597-411b-a54a-ed5c833e584d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:01.131011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadef31-2597-411b-a54a-ed5c833e584d None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:01.132722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 484/972] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:01.138902 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1638c4-19eb-4b69-a5ad-b1b000b2372a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:01.139190 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1638c4-19eb-4b69-a5ad-b1b000b2372a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:01.139471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1638c4-19eb-4b69-a5ad-b1b000b2372a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:01.158444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:01.158740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ae1638c4-19eb-4b69-a5ad-b1b000b2372a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:01.159982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ae1638c4-19eb-4b69-a5ad-b1b000b2372a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:01.203168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:01.208299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c8ea1-b9ec-4605-8dce-da39f61957ea None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:01.251112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 485/973] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:01.251667 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea24ee-2bbb-4c49-bd2a-4fae0395d1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:01.252026 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea24ee-2bbb-4c49-bd2a-4fae0395d1ba None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:01.252321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea24ee-2bbb-4c49-bd2a-4fae0395d1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:01.252591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea24ee-2bbb-4c49-bd2a-4fae0395d1ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:01.328243 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea24ee-2bbb-4c49-bd2a-4fae0395d1ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:01.329754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea24ee-2bbb-4c49-bd2a-4fae0395d1ba None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:01.394067 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:01.435765 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 489/974] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:47:01 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:47:01.443496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1638c4-19eb-4b69-a5ad-b1b000b2372a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:01.529986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea24ee-2bbb-4c49-bd2a-4fae0395d1ba None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:01.553803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1638c4-19eb-4b69-a5ad-b1b000b2372a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:01.588242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c723ba9-74f5-42af-8ddc-d6c563b7ee1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:01.588907 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c723ba9-74f5-42af-8ddc-d6c563b7ee1c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:01.589706 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c723ba9-74f5-42af-8ddc-d6c563b7ee1c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:01.590017 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c723ba9-74f5-42af-8ddc-d6c563b7ee1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:01.600631 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c723ba9-74f5-42af-8ddc-d6c563b7ee1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:01.602310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c723ba9-74f5-42af-8ddc-d6c563b7ee1c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:01.618673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2c723ba9-74f5-42af-8ddc-d6c563b7ee1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:47:01.618673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:01.618673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:01.618673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:01.618673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:01.618673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:01.618673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:01.618673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:01.618673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:01.618673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:01.618673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:01.618673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:01.618673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:47:01.618673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:47:01.618673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:47:01.618673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:47:01.619696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:01.619696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:01.619696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:01.619696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:01.619696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:01.619696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:01.619696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:01.619696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:01.619696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:01.619696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:01.619696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:01.619696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:01.619696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:01.619696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:01.619696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:01.619696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:01.619696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:47:01.620642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:47:01.620642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:47:01.620642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:01.620642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 486/975] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:01.625788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c723ba9-74f5-42af-8ddc-d6c563b7ee1c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:01.660160 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea24ee-2bbb-4c49-bd2a-4fae0395d1ba None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:01.670209 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:01.705352 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d4a1f-ded4-4d54-bad9-413450fbdb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:01.706119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d4a1f-ded4-4d54-bad9-413450fbdb9d None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:01.706402 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d4a1f-ded4-4d54-bad9-413450fbdb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:01.706675 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d4a1f-ded4-4d54-bad9-413450fbdb9d None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:01.707338 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d4a1f-ded4-4d54-bad9-413450fbdb9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:01.709154 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d4a1f-ded4-4d54-bad9-413450fbdb9d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:01.712567 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: WARNING keystone.server.flask.common [None req-a02d4a1f-ded4-4d54-bad9-413450fbdb9d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 14 11:47:01.733037 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c723ba9-74f5-42af-8ddc-d6c563b7ee1c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:01.744826 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.sql.core [None req-a02d4a1f-ded4-4d54-bad9-413450fbdb9d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 14 11:47:01.744826 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 14 11:47:01.744826 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [parameters: {'user_id': '9dfeab37bca640198038c0db849d4977', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 14 11:47:01.744826 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81582) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 14 11:47:01.762361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-a02d4a1f-ded4-4d54-bad9-413450fbdb9d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 14 11:47:01.762361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:01.762361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 11:47:01.762361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 11:47:01.762361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 11:47:01.762361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 11:47:01.762361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 11:47:01.762361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 11:47:01.762361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 11:47:01.762361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application conn.query(q) Sep 14 11:47:01.762361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 11:47:01.762361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 11:47:01.762361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 11:47:01.762361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result.read() Sep 14 11:47:01.762361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 11:47:01.763335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 11:47:01.763335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 11:47:01.763335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 11:47:01.763335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 11:47:01.763335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 11:47:01.763335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 11:47:01.763335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 11:47:01.763335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 14 11:47:01.763335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:01.763335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 14 11:47:01.763335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:01.763335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:01.763335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 14 11:47:01.763335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 14 11:47:01.763335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 14 11:47:01.763335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 14 11:47:01.763335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 11:47:01.763335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application next(self.gen) Sep 14 11:47:01.764316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 14 11:47:01.764316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application yield resource Sep 14 11:47:01.764316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 11:47:01.764316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application next(self.gen) Sep 14 11:47:01.764316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 14 11:47:01.764316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.session.rollback() Sep 14 11:47:01.764316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 14 11:47:01.764316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.force_reraise() Sep 14 11:47:01.764316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 14 11:47:01.764316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise self.value Sep 14 11:47:01.764316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 14 11:47:01.764316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 14 11:47:01.764316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 14 11:47:01.764316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application session.commit() Sep 14 11:47:01.764316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 14 11:47:01.764316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 14 11:47:01.764316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 14 11:47:01.765248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._prepare_impl() Sep 14 11:47:01.765248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 14 11:47:01.765248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.session.flush() Sep 14 11:47:01.765248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 14 11:47:01.765248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._flush(objects) Sep 14 11:47:01.765248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 14 11:47:01.765248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 14 11:47:01.765248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 14 11:47:01.765248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application compat.raise_( Sep 14 11:47:01.765248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 11:47:01.765248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception Sep 14 11:47:01.765248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 14 11:47:01.765248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application flush_context.execute() Sep 14 11:47:01.765248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 14 11:47:01.765248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rec.execute(self) Sep 14 11:47:01.765248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 14 11:47:01.765248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 14 11:47:01.767230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 14 11:47:01.767230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 14 11:47:01.767230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 14 11:47:01.767230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 14 11:47:01.767230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 14 11:47:01.767230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 14 11:47:01.767230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 14 11:47:01.767230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 14 11:47:01.767230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 14 11:47:01.767230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 14 11:47:01.767230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 14 11:47:01.767230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 14 11:47:01.767230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 14 11:47:01.767230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 14 11:47:01.767230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 11:47:01.767230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception Sep 14 11:47:01.769187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 11:47:01.769187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 11:47:01.769187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 11:47:01.769187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 11:47:01.769187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 11:47:01.769187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 11:47:01.769187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 11:47:01.769187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application conn.query(q) Sep 14 11:47:01.769187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 11:47:01.769187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 11:47:01.769187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 11:47:01.769187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result.read() Sep 14 11:47:01.769187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 11:47:01.769187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 11:47:01.769187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 11:47:01.769187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 11:47:01.773662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 11:47:01.773662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 11:47:01.773662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 11:47:01.773662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 11:47:01.773662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 14 11:47:01.773662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 14 11:47:01.773662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [parameters: {'user_id': '9dfeab37bca640198038c0db849d4977', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 14 11:47:01.773662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 14 11:47:01.773662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:01.773662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:47:01.773662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:01.773662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:01.773662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:01.773662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:01.773662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:01.774627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:01.774627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:01.774627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:01.774627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:01.774627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:01.774627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:01.774627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:01.774627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 14 11:47:01.774627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 14 11:47:01.774627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:01.774627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:01.774627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:47:01.774627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:47:01.774627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:47:01.774627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:47:01.774627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 14 11:47:01.774627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 14 11:47:01.775560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 14 11:47:01.775560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 14 11:47:01.775560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 14 11:47:01.775560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 14 11:47:01.775560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 14 11:47:01.775560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:01.775560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 490/976] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Sep 14 11:47:01 2022] POST /identity/v3/users => generated 160 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:47:01.775560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d4a1f-ded4-4d54-bad9-413450fbdb9d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:01.781056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932279f7-8154-4669-8584-94addd494f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:01.781918 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932279f7-8154-4669-8584-94addd494f46 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:01.782954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932279f7-8154-4669-8584-94addd494f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:01.783508 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932279f7-8154-4669-8584-94addd494f46 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:01.784411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932279f7-8154-4669-8584-94addd494f46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:01.785370 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932279f7-8154-4669-8584-94addd494f46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:01.787592 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932279f7-8154-4669-8584-94addd494f46 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:01.804487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:01.807976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 487/977] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:01.814441 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932279f7-8154-4669-8584-94addd494f46 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:01.889333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d4a1f-ded4-4d54-bad9-413450fbdb9d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:01.924713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:01.937235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60a50c-10da-4eef-ba2b-3782470d9160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:01.938362 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60a50c-10da-4eef-ba2b-3782470d9160 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:01.939135 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60a50c-10da-4eef-ba2b-3782470d9160 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:01.939898 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60a50c-10da-4eef-ba2b-3782470d9160 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:01.947984 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932279f7-8154-4669-8584-94addd494f46 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:01.949823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60a50c-10da-4eef-ba2b-3782470d9160 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:01.953605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60a50c-10da-4eef-ba2b-3782470d9160 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:01.970892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-ff60a50c-10da-4eef-ba2b-3782470d9160 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 11:47:01.970892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:01.970892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:01.970892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:01.970892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:01.970892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:01.970892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:01.970892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:01.970892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:01.970892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:01.970892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:01.970892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:01.970892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:47:01.970892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:47:01.970892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:47:01.970892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:47:01.971943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:01.971943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:01.971943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:47:01.971943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:47:01.971943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:47:01.971943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:47:01.971943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:01.971943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:01.971943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:01.971943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:01.971943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:01.971943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:01.971943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:01.971943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:01.971943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:01.971943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:01.971943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:01.971943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:01.973245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:01.973245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:01.973245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:47:01.973245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:47:01.973245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:47:01.973245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:47:01.973245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:47:01.973245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:47:01.973245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 11:47:01.973245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:01.976535 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 491/978] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:47:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:01.983141 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60a50c-10da-4eef-ba2b-3782470d9160 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:02.001454 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bec9b-da7a-4c95-a81c-b3543d46af2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:02.002581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bec9b-da7a-4c95-a81c-b3543d46af2b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:02.003051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bec9b-da7a-4c95-a81c-b3543d46af2b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:02.003505 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bec9b-da7a-4c95-a81c-b3543d46af2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:02.011555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9bec9b-da7a-4c95-a81c-b3543d46af2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:02.013347 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9bec9b-da7a-4c95-a81c-b3543d46af2b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:02.025952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-3d9bec9b-da7a-4c95-a81c-b3543d46af2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:47:02.025952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:02.025952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:02.025952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:02.025952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:02.025952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:02.025952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:02.025952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:02.025952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:02.025952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:02.025952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:02.025952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:02.025952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:47:02.025952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:47:02.025952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:47:02.026957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:47:02.026957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:02.026957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:02.026957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:02.026957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:02.026957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:02.026957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:02.026957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:02.026957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:02.026957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:02.026957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:02.026957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:02.026957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:02.026957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:02.026957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:02.026957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:02.026957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:02.027891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:47:02.027891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:47:02.027891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:47:02.027891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:47:02.027891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:47:02.027891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:47:02.027891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:47:02.027891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:02.030067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 488/979] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:47:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:02.035914 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bec9b-da7a-4c95-a81c-b3543d46af2b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:02.094197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:02.110556 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60a50c-10da-4eef-ba2b-3782470d9160 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:02.147286 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952d4f0-cea3-4982-bee2-a943edc8e8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:02.148094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952d4f0-cea3-4982-bee2-a943edc8e8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:02.148484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952d4f0-cea3-4982-bee2-a943edc8e8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:02.148878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952d4f0-cea3-4982-bee2-a943edc8e8d2 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:02.149717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952d4f0-cea3-4982-bee2-a943edc8e8d2 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:02.150151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952d4f0-cea3-4982-bee2-a943edc8e8d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:02.151700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952d4f0-cea3-4982-bee2-a943edc8e8d2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:02.160377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bec9b-da7a-4c95-a81c-b3543d46af2b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:02.196202 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 492/980] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 14 11:47:01 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:02.201259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c2b69-6733-4a41-875e-e2afe1150228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:02.202174 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c2b69-6733-4a41-875e-e2afe1150228 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:02.202780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c2b69-6733-4a41-875e-e2afe1150228 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:02.203472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c2b69-6733-4a41-875e-e2afe1150228 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:02.204386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c2b69-6733-4a41-875e-e2afe1150228 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:02.204854 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c2b69-6733-4a41-875e-e2afe1150228 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:02.206541 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c2b69-6733-4a41-875e-e2afe1150228 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:02.218299 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:02.223497 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 489/981] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:47:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:02.229538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952d4f0-cea3-4982-bee2-a943edc8e8d2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:02.348711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952d4f0-cea3-4982-bee2-a943edc8e8d2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:02.389301 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a617a7-0550-4b16-8201-14dbecbff1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:02.389971 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a617a7-0550-4b16-8201-14dbecbff1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:02.390258 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a617a7-0550-4b16-8201-14dbecbff1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:02.390537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a617a7-0550-4b16-8201-14dbecbff1a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:02.391275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a617a7-0550-4b16-8201-14dbecbff1a3 None admin] RBAC: Adding query filter params (role.id=09d8286638fb49d8a4499662066fda1a, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=d76a164ed7e14d9eb2db016b68a84d9a) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:02.391590 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a617a7-0550-4b16-8201-14dbecbff1a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:02.393025 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a617a7-0550-4b16-8201-14dbecbff1a3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:02.400285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 493/982] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:47:02 2022] GET /identity/v3/role_assignments?user.id=d76a164ed7e14d9eb2db016b68a84d9a&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=09d8286638fb49d8a4499662066fda1a => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:02.582078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5b987-d79e-425d-9004-0f58f837e93c None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:02.582576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5b987-d79e-425d-9004-0f58f837e93c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:02.582576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5b987-d79e-425d-9004-0f58f837e93c None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:02.584158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 490/983] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:02.590045 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83d51f-a7ef-49ae-a011-fb75c634452d None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:02.590339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83d51f-a7ef-49ae-a011-fb75c634452d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:02.590621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83d51f-a7ef-49ae-a011-fb75c634452d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:02.610323 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8e83d51f-a7ef-49ae-a011-fb75c634452d None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:02.611873 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8e83d51f-a7ef-49ae-a011-fb75c634452d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:02.613984 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:02.685906 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 494/984] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:02.957281 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5b987-d79e-425d-9004-0f58f837e93c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:03.065249 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5b987-d79e-425d-9004-0f58f837e93c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:03.100557 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb71b9-32f9-44bc-a557-493f7c8a9d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:03.101193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb71b9-32f9-44bc-a557-493f7c8a9d77 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:03.101478 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb71b9-32f9-44bc-a557-493f7c8a9d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:03.101755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb71b9-32f9-44bc-a557-493f7c8a9d77 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:03.102377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb71b9-32f9-44bc-a557-493f7c8a9d77 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:03.103751 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb71b9-32f9-44bc-a557-493f7c8a9d77 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:03.112744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-27fb71b9-32f9-44bc-a557-493f7c8a9d77 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 11:47:03.112744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:03.112744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:47:03.112744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:47:03.112744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:47:03.112744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:47:03.112744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:47:03.112744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:47:03.112744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 11:47:03.112744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:03.112744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:47:03.112744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:03.112744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:03.112744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:03.112744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:03.112744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:03.112744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:03.113774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:03.113774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:03.113774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:03.113774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:03.113774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:03.113774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:03.113774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:47:03.113774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:47:03.113774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:47:03.113774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:47:03.113774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:03.113774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:03.113774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:03.113774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:03.113774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:03.113774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:03.115008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:03.115008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:03.115008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:03.115008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:03.115008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:03.115008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:03.115008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:03.115008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:03.115008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:03.115008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:03.115008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:47:03.115008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:47:03.115008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 11:47:03.115008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:03.116220 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 491/985] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 14 11:47:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:03.119853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83d51f-a7ef-49ae-a011-fb75c634452d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:03.227592 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83d51f-a7ef-49ae-a011-fb75c634452d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:03.261731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00870c-3312-4576-bcc4-3743042f3c40 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:03.262081 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00870c-3312-4576-bcc4-3743042f3c40 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:03.262184 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc05bf-8c51-44e4-8fcb-41b3442fe649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:03.262627 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00870c-3312-4576-bcc4-3743042f3c40 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:03.262716 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc05bf-8c51-44e4-8fcb-41b3442fe649 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:03.262919 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc05bf-8c51-44e4-8fcb-41b3442fe649 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:03.263191 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc05bf-8c51-44e4-8fcb-41b3442fe649 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:03.263811 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 492/986] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:03.263926 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc05bf-8c51-44e4-8fcb-41b3442fe649 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:03.264107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc05bf-8c51-44e4-8fcb-41b3442fe649 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:03.265466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc05bf-8c51-44e4-8fcb-41b3442fe649 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:03.268743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb673be1-cabf-4d74-9f24-503649b722e1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:03.269026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb673be1-cabf-4d74-9f24-503649b722e1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:03.269301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb673be1-cabf-4d74-9f24-503649b722e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:03.274122 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 495/987] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 14 11:47:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:03.279434 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc05bf-8c51-44e4-8fcb-41b3442fe649 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:03.282198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:03.294642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-cb673be1-cabf-4d74-9f24-503649b722e1 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:03.297333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cb673be1-cabf-4d74-9f24-503649b722e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:03.375701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:03.384055 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 493/988] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 11:47:03.384449 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc05bf-8c51-44e4-8fcb-41b3442fe649 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:03.418498 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a84a1-819c-4ea0-8273-d91f3d0dcc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:03.419101 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a84a1-819c-4ea0-8273-d91f3d0dcc3b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:03.419381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a84a1-819c-4ea0-8273-d91f3d0dcc3b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:03.419653 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a84a1-819c-4ea0-8273-d91f3d0dcc3b None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:03.420277 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a84a1-819c-4ea0-8273-d91f3d0dcc3b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:03.420581 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a84a1-819c-4ea0-8273-d91f3d0dcc3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:03.421950 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a84a1-819c-4ea0-8273-d91f3d0dcc3b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:03.427142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 496/989] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 14 11:47:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:03.432783 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb673be1-cabf-4d74-9f24-503649b722e1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:03.533831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb673be1-cabf-4d74-9f24-503649b722e1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:03.569036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26f945-47c4-402a-bc59-70fa62f2c81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:03.569639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26f945-47c4-402a-bc59-70fa62f2c81e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:03.569959 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26f945-47c4-402a-bc59-70fa62f2c81e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:03.570240 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26f945-47c4-402a-bc59-70fa62f2c81e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:03.570940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26f945-47c4-402a-bc59-70fa62f2c81e None admin] RBAC: Adding query filter params (interface=admin, service_id=bdc188a3ec984d5da35a7d7a047e0abf, region_id=RegionOne) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:03.571254 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26f945-47c4-402a-bc59-70fa62f2c81e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:03.572614 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26f945-47c4-402a-bc59-70fa62f2c81e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:03.577852 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 494/990] 10.222.0.62 () {56 vars in 1249 bytes} [Wed Sep 14 11:47:03 2022] GET /identity/v3/endpoints?service_id=bdc188a3ec984d5da35a7d7a047e0abf&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:03.583327 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a84a1-819c-4ea0-8273-d91f3d0dcc3b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:03.650016 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:03.661579 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26f945-47c4-402a-bc59-70fa62f2c81e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:03.687642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a84a1-819c-4ea0-8273-d91f3d0dcc3b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:03.723023 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734ecec-1e7f-4c21-9da5-7ec0d822d636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:03.723705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734ecec-1e7f-4c21-9da5-7ec0d822d636 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:03.723840 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:03.724013 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734ecec-1e7f-4c21-9da5-7ec0d822d636 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:03.724293 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734ecec-1e7f-4c21-9da5-7ec0d822d636 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:03.724920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734ecec-1e7f-4c21-9da5-7ec0d822d636 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:03.726323 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734ecec-1e7f-4c21-9da5-7ec0d822d636 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:03.733313 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 497/991] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Sep 14 11:47:03 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 11:47:03.776430 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26f945-47c4-402a-bc59-70fa62f2c81e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:03.815641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9b59f-f486-4ec3-a0e0-c34a3ba485f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:03.816207 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9b59f-f486-4ec3-a0e0-c34a3ba485f6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:03.816490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9b59f-f486-4ec3-a0e0-c34a3ba485f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:03.816767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9b59f-f486-4ec3-a0e0-c34a3ba485f6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:03.817368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9b59f-f486-4ec3-a0e0-c34a3ba485f6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:03.821224 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9b59f-f486-4ec3-a0e0-c34a3ba485f6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:03.831500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2ee9b59f-f486-4ec3-a0e0-c34a3ba485f6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 11:47:03.831500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:03.831500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:47:03.831500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:47:03.831500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:47:03.831500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:47:03.831500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:47:03.831500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:47:03.831500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 11:47:03.831500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:03.831500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:47:03.831500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:03.831500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:03.831500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:03.831500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:03.831500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:03.831500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:03.832581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:03.832581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:03.832581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:03.832581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:03.832581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:03.832581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:03.832581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:47:03.832581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:47:03.832581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:47:03.832581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:47:03.832581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:03.832581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:03.832581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:03.832581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:03.832581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:03.832581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:03.833446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:03.833446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:03.833446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:03.833446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:03.833446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:03.833446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:03.833446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:03.833446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:03.833446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:03.833446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:03.833446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:47:03.833446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:47:03.833446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 11:47:03.833446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:03.833446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 495/992] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:47:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:47:03.838686 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734ecec-1e7f-4c21-9da5-7ec0d822d636 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:03.950254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734ecec-1e7f-4c21-9da5-7ec0d822d636 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:03.980901 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:03.986687 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa48757-f311-40ba-858e-9453d584336b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:03.987305 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa48757-f311-40ba-858e-9453d584336b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:03.987587 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa48757-f311-40ba-858e-9453d584336b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:03.987862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa48757-f311-40ba-858e-9453d584336b None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:03.988493 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa48757-f311-40ba-858e-9453d584336b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:03.988857 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa48757-f311-40ba-858e-9453d584336b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:03.990251 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa48757-f311-40ba-858e-9453d584336b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:03.995228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 498/993] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 14 11:47:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:04.001236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9b59f-f486-4ec3-a0e0-c34a3ba485f6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:04.105357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9b59f-f486-4ec3-a0e0-c34a3ba485f6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:04.128576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:04.141168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a2c42-16a2-4a91-8ec0-9ed9b62ba5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:04.143339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a2c42-16a2-4a91-8ec0-9ed9b62ba5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:04.143339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a2c42-16a2-4a91-8ec0-9ed9b62ba5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:04.143339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a2c42-16a2-4a91-8ec0-9ed9b62ba5c1 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:04.143339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a2c42-16a2-4a91-8ec0-9ed9b62ba5c1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:04.143339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a2c42-16a2-4a91-8ec0-9ed9b62ba5c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:04.144725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a2c42-16a2-4a91-8ec0-9ed9b62ba5c1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:04.149694 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 496/994] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 14 11:47:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:04.156097 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa48757-f311-40ba-858e-9453d584336b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:04.259137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa48757-f311-40ba-858e-9453d584336b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:04.293566 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faf376-2a7e-41d5-83d6-993ca5d570f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:04.294231 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faf376-2a7e-41d5-83d6-993ca5d570f3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:04.294528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faf376-2a7e-41d5-83d6-993ca5d570f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:04.294809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faf376-2a7e-41d5-83d6-993ca5d570f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:04.295500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77faf376-2a7e-41d5-83d6-993ca5d570f3 None admin] RBAC: Adding query filter params (interface=public, service_id=3c8b0e3ea7324d858753b34e62fd242f, region_id=RegionOne) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:04.295814 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77faf376-2a7e-41d5-83d6-993ca5d570f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:04.297168 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77faf376-2a7e-41d5-83d6-993ca5d570f3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:04.302692 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 499/995] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Sep 14 11:47:04 2022] GET /identity/v3/endpoints?service_id=3c8b0e3ea7324d858753b34e62fd242f&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:04.308220 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a2c42-16a2-4a91-8ec0-9ed9b62ba5c1 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:04.408513 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a2c42-16a2-4a91-8ec0-9ed9b62ba5c1 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:04.442751 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489c6c3-a958-45b5-82ec-cc8e8ea7da01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:04.443365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489c6c3-a958-45b5-82ec-cc8e8ea7da01 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:04.443650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489c6c3-a958-45b5-82ec-cc8e8ea7da01 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:04.443928 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489c6c3-a958-45b5-82ec-cc8e8ea7da01 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:04.444542 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489c6c3-a958-45b5-82ec-cc8e8ea7da01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:04.445976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489c6c3-a958-45b5-82ec-cc8e8ea7da01 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:04.461890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 497/996] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Sep 14 11:47:04 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 11:47:04.601352 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc7b4d-0a7a-4391-b0a0-bd5c0ff7e489 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:04.601682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc7b4d-0a7a-4391-b0a0-bd5c0ff7e489 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:04.601941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc7b4d-0a7a-4391-b0a0-bd5c0ff7e489 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:04.603488 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 500/997] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:04.609319 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f19de-f83c-47e8-886c-23ac64203c1d None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:04.609660 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f19de-f83c-47e8-886c-23ac64203c1d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:04.609905 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f19de-f83c-47e8-886c-23ac64203c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:04.630601 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-fe0f19de-f83c-47e8-886c-23ac64203c1d None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:04.631807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fe0f19de-f83c-47e8-886c-23ac64203c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:04.680659 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:04.703411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 498/998] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:04.973144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc7b4d-0a7a-4391-b0a0-bd5c0ff7e489 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:05.035771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:05.079643 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc7b4d-0a7a-4391-b0a0-bd5c0ff7e489 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:05.112739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525204a5-66a6-4f07-9c66-c70ce6861d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:05.113370 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525204a5-66a6-4f07-9c66-c70ce6861d9b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:05.113654 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525204a5-66a6-4f07-9c66-c70ce6861d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:05.113956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525204a5-66a6-4f07-9c66-c70ce6861d9b None admin] PATH_INFO: `/v3/services/image` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:05.114540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525204a5-66a6-4f07-9c66-c70ce6861d9b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:05.116024 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525204a5-66a6-4f07-9c66-c70ce6861d9b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:05.125395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-525204a5-66a6-4f07-9c66-c70ce6861d9b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 11:47:05.125395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:05.125395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:47:05.125395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:47:05.125395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:47:05.125395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:47:05.125395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:47:05.125395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:47:05.125395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 11:47:05.125395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:05.125395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:47:05.125395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:05.125395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:05.125395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:05.125395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:05.125395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:05.125395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:05.126591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:05.126591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:05.126591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:05.126591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:05.126591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:05.126591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:05.126591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:47:05.126591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:47:05.126591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:47:05.126591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:47:05.126591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:05.126591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:05.126591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:05.126591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:05.126591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:05.126591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:05.127458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:05.127458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:05.127458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:05.127458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:05.127458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:05.127458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:05.127458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:05.127458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:05.127458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:05.127458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:05.127458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:47:05.127458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:47:05.127458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 11:47:05.127458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:05.130185 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 501/999] 10.222.0.62 () {56 vars in 1105 bytes} [Wed Sep 14 11:47:04 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:05.132316 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f19de-f83c-47e8-886c-23ac64203c1d None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:05.239381 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f19de-f83c-47e8-886c-23ac64203c1d None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:05.273994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2203b-a18e-47a8-8f77-d835facd60fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:05.274609 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2203b-a18e-47a8-8f77-d835facd60fa None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:05.274895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2203b-a18e-47a8-8f77-d835facd60fa None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:05.275176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2203b-a18e-47a8-8f77-d835facd60fa None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:05.275811 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec2203b-a18e-47a8-8f77-d835facd60fa None admin] RBAC: Adding query filter params (name=image) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:05.276119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec2203b-a18e-47a8-8f77-d835facd60fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:05.278058 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec2203b-a18e-47a8-8f77-d835facd60fa None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:05.283127 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 499/1000] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:05.288533 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525204a5-66a6-4f07-9c66-c70ce6861d9b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:05.391009 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525204a5-66a6-4f07-9c66-c70ce6861d9b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:05.413173 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:05.428451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64e87c-5eab-4559-a700-d7f637cded14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:05.429062 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64e87c-5eab-4559-a700-d7f637cded14 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:05.429343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64e87c-5eab-4559-a700-d7f637cded14 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:05.432398 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64e87c-5eab-4559-a700-d7f637cded14 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:05.432398 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64e87c-5eab-4559-a700-d7f637cded14 None admin] RBAC: Adding query filter params (type=image) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:05.432398 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64e87c-5eab-4559-a700-d7f637cded14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:05.432398 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64e87c-5eab-4559-a700-d7f637cded14 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:05.442084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 502/1001] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:05 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:05.896212 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a6209-f798-4133-9ecf-ef3998649bea None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:05.896728 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a6209-f798-4133-9ecf-ef3998649bea None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:05.896809 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a6209-f798-4133-9ecf-ef3998649bea None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:05.898272 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 500/1002] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:05.904738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a1949-e48b-4c8a-8f91-caede9b24173 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:05.904738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a1949-e48b-4c8a-8f91-caede9b24173 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:05.904738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a1949-e48b-4c8a-8f91-caede9b24173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:05.922222 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a26a1949-e48b-4c8a-8f91-caede9b24173 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:05.923649 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a26a1949-e48b-4c8a-8f91-caede9b24173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:05.991805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 503/1003] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:05 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:06.254958 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a6209-f798-4133-9ecf-ef3998649bea None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:06.304498 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:06.361062 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a6209-f798-4133-9ecf-ef3998649bea None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:06.395224 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863e775-be23-4674-a925-2a747e588bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:06.395855 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863e775-be23-4674-a925-2a747e588bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:06.396139 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863e775-be23-4674-a925-2a747e588bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:06.396416 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863e775-be23-4674-a925-2a747e588bc5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:06.396998 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863e775-be23-4674-a925-2a747e588bc5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:06.398418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863e775-be23-4674-a925-2a747e588bc5 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:06.407830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-9863e775-be23-4674-a925-2a747e588bc5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 11:47:06.407830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:06.407830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:47:06.407830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:47:06.407830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:47:06.407830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:47:06.407830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:47:06.407830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:47:06.407830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 11:47:06.407830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:06.407830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:47:06.407830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:06.407830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:06.407830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:06.407830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:06.407830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:06.407830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:06.408884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:06.408884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:06.408884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:06.408884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:06.408884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:06.408884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:06.408884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:47:06.408884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:47:06.408884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:47:06.408884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:47:06.408884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:06.408884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:06.408884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:06.408884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:06.408884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:06.408884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:06.409818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:06.409818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:06.409818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:06.409818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:06.409818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:06.409818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:06.409818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:06.409818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:06.409818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:06.409818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:06.409818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:47:06.409818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:47:06.409818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 11:47:06.409818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:06.413945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 501/1004] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 14 11:47:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:06.414765 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a1949-e48b-4c8a-8f91-caede9b24173 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:06.516731 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a1949-e48b-4c8a-8f91-caede9b24173 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:06.533662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:06.551300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674840e-1d92-45a3-a914-14a982459897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:06.551914 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674840e-1d92-45a3-a914-14a982459897 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:06.552194 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674840e-1d92-45a3-a914-14a982459897 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:06.552469 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674840e-1d92-45a3-a914-14a982459897 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:06.553094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674840e-1d92-45a3-a914-14a982459897 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:06.553399 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674840e-1d92-45a3-a914-14a982459897 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:06.554867 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674840e-1d92-45a3-a914-14a982459897 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:06.559706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 504/1005] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 14 11:47:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:06.565144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863e775-be23-4674-a925-2a747e588bc5 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:06.598801 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f52c8-1bd9-4324-b857-33ab03749249 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:06.599127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f52c8-1bd9-4324-b857-33ab03749249 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:06.599354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f52c8-1bd9-4324-b857-33ab03749249 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:06.600809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 505/1006] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:47:06.605649 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5c339-be53-413d-b7ad-f8017ad84ad2 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:06.605966 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5c339-be53-413d-b7ad-f8017ad84ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:06.606229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5c339-be53-413d-b7ad-f8017ad84ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:06.624605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-71c5c339-be53-413d-b7ad-f8017ad84ad2 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:06.625768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-71c5c339-be53-413d-b7ad-f8017ad84ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:06.680617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863e775-be23-4674-a925-2a747e588bc5 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:06.690325 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:06.697847 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 506/1007] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:06 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 11:47:06.718373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa994b-ea1c-4a0d-9a13-b5d39d3ea44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:06.718991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa994b-ea1c-4a0d-9a13-b5d39d3ea44c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:06.719298 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa994b-ea1c-4a0d-9a13-b5d39d3ea44c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:06.719600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa994b-ea1c-4a0d-9a13-b5d39d3ea44c None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:06.720235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa994b-ea1c-4a0d-9a13-b5d39d3ea44c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:06.720545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa994b-ea1c-4a0d-9a13-b5d39d3ea44c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:06.721992 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa994b-ea1c-4a0d-9a13-b5d39d3ea44c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:06.731988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 502/1008] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 14 11:47:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:06.732980 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5c339-be53-413d-b7ad-f8017ad84ad2 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:06.841145 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5c339-be53-413d-b7ad-f8017ad84ad2 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:06.852358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:06.875100 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08256cf-c8d1-4914-8db0-accbe9536611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:06.875754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08256cf-c8d1-4914-8db0-accbe9536611 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:06.876041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08256cf-c8d1-4914-8db0-accbe9536611 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:06.876318 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08256cf-c8d1-4914-8db0-accbe9536611 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:06.876980 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08256cf-c8d1-4914-8db0-accbe9536611 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:06.878360 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08256cf-c8d1-4914-8db0-accbe9536611 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:06.966662 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa994b-ea1c-4a0d-9a13-b5d39d3ea44c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:06.970952 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 507/1009] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 11:47:06 2022] POST /identity/v3/endpoints => generated 325 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:47:07.090078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa994b-ea1c-4a0d-9a13-b5d39d3ea44c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:07.134222 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28fbf17-9b40-4910-8270-b31da15263ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:07.134962 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28fbf17-9b40-4910-8270-b31da15263ea None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:07.136426 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28fbf17-9b40-4910-8270-b31da15263ea None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:07.136426 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28fbf17-9b40-4910-8270-b31da15263ea None admin] PATH_INFO: `/v3/services/compute` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:07.136426 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28fbf17-9b40-4910-8270-b31da15263ea None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:07.137893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28fbf17-9b40-4910-8270-b31da15263ea None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:07.147494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-e28fbf17-9b40-4910-8270-b31da15263ea None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 11:47:07.147494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:07.147494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:47:07.147494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:47:07.147494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:47:07.147494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:47:07.147494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:47:07.147494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:47:07.147494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 11:47:07.147494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:07.147494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:47:07.147494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:07.147494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:07.147494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:07.147494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:07.147494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:07.147494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:07.148778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:07.148778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:07.148778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:07.148778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:07.148778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:07.148778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:07.148778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:47:07.148778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:47:07.148778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:47:07.148778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:47:07.148778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:07.148778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:07.148778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:07.148778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:07.148778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:07.148778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:07.149839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:07.149839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:07.149839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:07.149839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:07.149839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:07.149839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:07.149839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:07.149839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:07.149839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:07.149839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:07.149839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:47:07.149839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:47:07.149839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 11:47:07.149839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:07.154231 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 503/1010] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:47:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:47:07.156018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08256cf-c8d1-4914-8db0-accbe9536611 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:07.263660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08256cf-c8d1-4914-8db0-accbe9536611 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:07.298573 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86eead-7bec-4817-a9d6-d2c3ae0cf11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:07.299353 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86eead-7bec-4817-a9d6-d2c3ae0cf11e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:07.299738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86eead-7bec-4817-a9d6-d2c3ae0cf11e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:07.300115 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86eead-7bec-4817-a9d6-d2c3ae0cf11e None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:07.300847 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86eead-7bec-4817-a9d6-d2c3ae0cf11e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:07.301255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86eead-7bec-4817-a9d6-d2c3ae0cf11e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:07.302745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86eead-7bec-4817-a9d6-d2c3ae0cf11e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:07.312643 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 508/1011] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 14 11:47:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:07.314041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28fbf17-9b40-4910-8270-b31da15263ea None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:07.346793 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:07.415486 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28fbf17-9b40-4910-8270-b31da15263ea None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:07.451135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c50e1-5dac-4890-ab56-d976cf21986e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:07.451739 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c50e1-5dac-4890-ab56-d976cf21986e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:07.452022 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c50e1-5dac-4890-ab56-d976cf21986e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:07.452287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c50e1-5dac-4890-ab56-d976cf21986e None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:07.452916 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c50e1-5dac-4890-ab56-d976cf21986e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:07.453213 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c50e1-5dac-4890-ab56-d976cf21986e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:07.454602 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c50e1-5dac-4890-ab56-d976cf21986e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:07.459560 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 504/1012] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 14 11:47:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:07.465506 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86eead-7bec-4817-a9d6-d2c3ae0cf11e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:07.505325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:07.570467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86eead-7bec-4817-a9d6-d2c3ae0cf11e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:07.597083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a956abd-4dd4-4357-9eaa-6c0c74747b53 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:07.597354 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a956abd-4dd4-4357-9eaa-6c0c74747b53 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:07.597566 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a956abd-4dd4-4357-9eaa-6c0c74747b53 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:07.599141 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 505/1013] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:47:07.604519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68bb56-f76e-4dee-8676-21b66ab1b462 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:07.605348 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68bb56-f76e-4dee-8676-21b66ab1b462 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:07.605623 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68bb56-f76e-4dee-8676-21b66ab1b462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:07.608272 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45a5eb-4af9-4044-8b11-79c22057a48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:07.608886 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45a5eb-4af9-4044-8b11-79c22057a48a None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:07.609166 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45a5eb-4af9-4044-8b11-79c22057a48a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:07.609437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45a5eb-4af9-4044-8b11-79c22057a48a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:07.610157 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45a5eb-4af9-4044-8b11-79c22057a48a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:07.611581 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45a5eb-4af9-4044-8b11-79c22057a48a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:07.623740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4e68bb56-f76e-4dee-8676-21b66ab1b462 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:07.625018 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4e68bb56-f76e-4dee-8676-21b66ab1b462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:07.691320 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 506/1014] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:07 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 11:47:07.699833 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 509/1015] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 11:47:07 2022] POST /identity/v3/endpoints => generated 334 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:47:07.961218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68bb56-f76e-4dee-8676-21b66ab1b462 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:08.025467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:08.098249 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68bb56-f76e-4dee-8676-21b66ab1b462 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:08.137823 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7abf14-20bd-43d0-a9be-0d66f8898277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:08.138456 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7abf14-20bd-43d0-a9be-0d66f8898277 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:08.138736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7abf14-20bd-43d0-a9be-0d66f8898277 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:08.139008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7abf14-20bd-43d0-a9be-0d66f8898277 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:08.139666 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7abf14-20bd-43d0-a9be-0d66f8898277 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:08.141021 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7abf14-20bd-43d0-a9be-0d66f8898277 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:08.231511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 507/1016] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 14 11:47:07 2022] POST /identity/v3/services => generated 242 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:47:09.174411 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128e619-34fd-4531-b1b0-d25f46deacb2 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:09.174969 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128e619-34fd-4531-b1b0-d25f46deacb2 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:09.174969 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128e619-34fd-4531-b1b0-d25f46deacb2 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:09.176329 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 510/1017] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:09.181392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d798f-b946-450f-913e-e38a3b195cab None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:09.181775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d798f-b946-450f-913e-e38a3b195cab None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:09.181973 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d798f-b946-450f-913e-e38a3b195cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:09.201029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-6e5d798f-b946-450f-913e-e38a3b195cab None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:09.202235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6e5d798f-b946-450f-913e-e38a3b195cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:09.274736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 508/1018] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:09.539461 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128e619-34fd-4531-b1b0-d25f46deacb2 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:09.631215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:09.641760 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128e619-34fd-4531-b1b0-d25f46deacb2 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:09.675137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a169a4b-8186-46e4-b238-a370833debb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:09.675958 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a169a4b-8186-46e4-b238-a370833debb7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:09.676640 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a169a4b-8186-46e4-b238-a370833debb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:09.676960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a169a4b-8186-46e4-b238-a370833debb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:09.678171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a169a4b-8186-46e4-b238-a370833debb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:09.679594 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a169a4b-8186-46e4-b238-a370833debb7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:09.681348 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 511/1019] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:09.687380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d798f-b946-450f-913e-e38a3b195cab None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:09.697921 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:09.786801 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d798f-b946-450f-913e-e38a3b195cab None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:09.825589 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26727d8-5237-4acc-ad53-1dc4a396659a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:09.828940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26727d8-5237-4acc-ad53-1dc4a396659a None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:09.828940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26727d8-5237-4acc-ad53-1dc4a396659a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:09.828940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26727d8-5237-4acc-ad53-1dc4a396659a None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:09.828940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26727d8-5237-4acc-ad53-1dc4a396659a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:09.828940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26727d8-5237-4acc-ad53-1dc4a396659a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:09.897888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 509/1020] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 14 11:47:09 2022] POST /identity/v3/projects => generated 311 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 11:47:09.971847 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58dcfd5-65d5-4bab-b208-57b410d7e117 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:09.972359 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58dcfd5-65d5-4bab-b208-57b410d7e117 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:09.972546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58dcfd5-65d5-4bab-b208-57b410d7e117 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:09.974159 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 512/1021] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:09.980302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fd21a-2a6c-40ad-8e7e-7fc3c3eae141 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:09.980592 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fd21a-2a6c-40ad-8e7e-7fc3c3eae141 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:09.980878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fd21a-2a6c-40ad-8e7e-7fc3c3eae141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:10.000254 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c26fd21a-2a6c-40ad-8e7e-7fc3c3eae141 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:10.001518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c26fd21a-2a6c-40ad-8e7e-7fc3c3eae141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:10.064252 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:10.078945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 510/1022] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:10.350471 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58dcfd5-65d5-4bab-b208-57b410d7e117 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:10.391047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65557a42-84cd-4686-b863-82a543ab476c None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:10.391567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65557a42-84cd-4686-b863-82a543ab476c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:10.391647 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65557a42-84cd-4686-b863-82a543ab476c None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:10.393104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 511/1023] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:47:10.397938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62948251-7c24-448d-b362-f0a777f2f4a7 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:10.398225 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62948251-7c24-448d-b362-f0a777f2f4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:10.398498 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62948251-7c24-448d-b362-f0a777f2f4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:10.417629 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-62948251-7c24-448d-b362-f0a777f2f4a7 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:10.418840 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-62948251-7c24-448d-b362-f0a777f2f4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:10.458826 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58dcfd5-65d5-4bab-b208-57b410d7e117 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:10.495216 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301e7ad-df5e-42c8-852a-77eefd350ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:10.495844 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301e7ad-df5e-42c8-852a-77eefd350ebc None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:10.496122 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301e7ad-df5e-42c8-852a-77eefd350ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:10.496393 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301e7ad-df5e-42c8-852a-77eefd350ebc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:10.498311 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 512/1024] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 11:47:10.506877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f301e7ad-df5e-42c8-852a-77eefd350ebc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:47:10.511709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-f301e7ad-df5e-42c8-852a-77eefd350ebc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 11:47:10.511709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:10.511709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:10.511709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:10.511709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:10.511709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:10.511709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:10.511709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:10.511709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:10.511709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:10.511709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:10.511709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:10.511709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:47:10.511709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:47:10.511709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:47:10.511709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:47:10.512651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:47:10.512651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:47:10.512651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:47:10.512651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:47:10.512651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:10.512651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:10.512651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:10.512651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:10.512651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:10.512651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:10.512651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:10.512651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:10.512651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:10.512651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:10.512651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:10.512651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:10.513515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:10.513515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:10.513515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:10.513515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:10.513515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:47:10.513515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:47:10.513515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:47:10.513515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:47:10.513515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:47:10.513515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:47:10.513515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 11:47:10.513515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:10.517953 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 513/1025] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Sep 14 11:47:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:10.518971 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62948251-7c24-448d-b362-f0a777f2f4a7 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:10.621572 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62948251-7c24-448d-b362-f0a777f2f4a7 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:10.641623 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:10.661294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e36b1-c03c-4a4f-a651-83a903938d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:10.661945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e36b1-c03c-4a4f-a651-83a903938d6a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:10.662233 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e36b1-c03c-4a4f-a651-83a903938d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:10.662508 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e36b1-c03c-4a4f-a651-83a903938d6a None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:10.663181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e36b1-c03c-4a4f-a651-83a903938d6a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:10.663547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e36b1-c03c-4a4f-a651-83a903938d6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:10.664920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e36b1-c03c-4a4f-a651-83a903938d6a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:10.677892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 513/1026] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Sep 14 11:47:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:10.680549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301e7ad-df5e-42c8-852a-77eefd350ebc None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:10.701002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:10.760178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e36b1-c03c-4a4f-a651-83a903938d6a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:10.787826 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301e7ad-df5e-42c8-852a-77eefd350ebc None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:10.835700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284df5d3-888f-41c0-b003-0e8c7e5fc911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:10.836344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284df5d3-888f-41c0-b003-0e8c7e5fc911 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:10.836673 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284df5d3-888f-41c0-b003-0e8c7e5fc911 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:10.836955 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284df5d3-888f-41c0-b003-0e8c7e5fc911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:10.848299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284df5d3-888f-41c0-b003-0e8c7e5fc911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:10.850206 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284df5d3-888f-41c0-b003-0e8c7e5fc911 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:10.866435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-284df5d3-888f-41c0-b003-0e8c7e5fc911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:47:10.866435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:10.866435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:10.866435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:10.866435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:10.866435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:10.866435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:10.866435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:10.866435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:10.866435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:10.866435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:10.866435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:10.866435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:47:10.866435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:47:10.866435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:47:10.866435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:47:10.867381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:10.867381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:10.867381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:10.867381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:10.867381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:10.867381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:10.867381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:10.867381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:10.867381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:10.867381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:10.867381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:10.867381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:10.867381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:10.867381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:10.867381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:10.867381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:10.867381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:47:10.868289 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:47:10.868289 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:47:10.868289 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:10.868289 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 514/1027] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:10.873431 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284df5d3-888f-41c0-b003-0e8c7e5fc911 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:10.875848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e36b1-c03c-4a4f-a651-83a903938d6a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:10.912186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c056e97-ac51-4604-85b1-0f36bd148783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:10.912811 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c056e97-ac51-4604-85b1-0f36bd148783 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:10.913092 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c056e97-ac51-4604-85b1-0f36bd148783 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:10.913365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c056e97-ac51-4604-85b1-0f36bd148783 None admin] PATH_INFO: `/v3/services/image` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:10.913970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c056e97-ac51-4604-85b1-0f36bd148783 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:10.915331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c056e97-ac51-4604-85b1-0f36bd148783 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:10.924397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2c056e97-ac51-4604-85b1-0f36bd148783 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 11:47:10.924397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:10.924397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:47:10.924397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:47:10.924397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:47:10.924397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:47:10.924397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:47:10.924397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:47:10.924397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 11:47:10.924397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:10.924397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:47:10.924397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:10.924397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:10.924397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:10.924397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:10.924397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:10.924397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:10.925402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:10.925402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:10.925402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:10.925402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:10.925402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:10.925402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:10.925402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:47:10.925402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:47:10.925402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:47:10.925402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:47:10.925402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:10.925402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:10.925402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:10.925402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:10.925402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:10.925402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:10.926296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:10.926296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:10.926296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:10.926296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:10.926296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:10.926296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:10.926296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:10.926296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:10.926296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:10.926296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:10.926296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:47:10.926296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:47:10.926296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 11:47:10.926296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:10.926296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 514/1028] 10.222.0.62 () {56 vars in 1105 bytes} [Wed Sep 14 11:47:10 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:47:10.931170 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c056e97-ac51-4604-85b1-0f36bd148783 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:10.951139 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:10.981491 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:10.982959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284df5d3-888f-41c0-b003-0e8c7e5fc911 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:11.018377 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c59b88-4314-40ce-9808-751fdbc5eaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:11.018989 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c59b88-4314-40ce-9808-751fdbc5eaaf None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:11.019276 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c59b88-4314-40ce-9808-751fdbc5eaaf None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:11.019553 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c59b88-4314-40ce-9808-751fdbc5eaaf None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:11.020242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c59b88-4314-40ce-9808-751fdbc5eaaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:11.020552 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c59b88-4314-40ce-9808-751fdbc5eaaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:11.021929 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c59b88-4314-40ce-9808-751fdbc5eaaf None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:11.033952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c056e97-ac51-4604-85b1-0f36bd148783 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:11.034772 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 515/1029] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:11.040318 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c59b88-4314-40ce-9808-751fdbc5eaaf None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:11.077651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62dbaa-8e6b-4be5-a5fd-cb5d24cdf795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:11.079046 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62dbaa-8e6b-4be5-a5fd-cb5d24cdf795 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:11.079327 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62dbaa-8e6b-4be5-a5fd-cb5d24cdf795 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:11.079600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62dbaa-8e6b-4be5-a5fd-cb5d24cdf795 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:11.080235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62dbaa-8e6b-4be5-a5fd-cb5d24cdf795 None admin] RBAC: Adding query filter params (name=image) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:11.080540 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62dbaa-8e6b-4be5-a5fd-cb5d24cdf795 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:11.081927 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62dbaa-8e6b-4be5-a5fd-cb5d24cdf795 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:11.088185 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 515/1030] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:11.094148 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62dbaa-8e6b-4be5-a5fd-cb5d24cdf795 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:11.151051 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c59b88-4314-40ce-9808-751fdbc5eaaf None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:11.182349 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:11.189756 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1818e16-95de-41d6-a2e9-40d58154404d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:11.190529 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1818e16-95de-41d6-a2e9-40d58154404d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:11.190810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1818e16-95de-41d6-a2e9-40d58154404d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:11.191176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1818e16-95de-41d6-a2e9-40d58154404d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:11.198816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62dbaa-8e6b-4be5-a5fd-cb5d24cdf795 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:11.199155 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1818e16-95de-41d6-a2e9-40d58154404d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:11.200659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1818e16-95de-41d6-a2e9-40d58154404d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:11.209690 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:11.213634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-b1818e16-95de-41d6-a2e9-40d58154404d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 11:47:11.213634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:11.213634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:11.213634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:11.213634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:11.213634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:11.213634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:11.213634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:11.213634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:11.213634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:11.213634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:11.213634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:11.213634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:47:11.213634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:47:11.213634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:47:11.213634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:47:11.215190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:11.215190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:11.215190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:47:11.215190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:47:11.215190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:47:11.215190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:47:11.215190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:11.215190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:11.215190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:11.215190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:11.215190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:11.215190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:11.215190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:11.215190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:11.215190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:11.215190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:11.215190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:11.215190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:11.216171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:11.216171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:11.216171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:47:11.216171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:47:11.216171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:47:11.216171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:47:11.216171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:47:11.216171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:47:11.216171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 11:47:11.216171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:11.223319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 516/1031] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Sep 14 11:47:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:47:11.224368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1818e16-95de-41d6-a2e9-40d58154404d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:11.234539 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5baa3-88d8-4af1-b562-6c96473a63c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:11.235150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5baa3-88d8-4af1-b562-6c96473a63c4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:11.235434 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5baa3-88d8-4af1-b562-6c96473a63c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:11.235712 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5baa3-88d8-4af1-b562-6c96473a63c4 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:11.236409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5baa3-88d8-4af1-b562-6c96473a63c4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:11.236719 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5baa3-88d8-4af1-b562-6c96473a63c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:11.238109 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5baa3-88d8-4af1-b562-6c96473a63c4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:11.247628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 516/1032] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:11.248687 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5baa3-88d8-4af1-b562-6c96473a63c4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:11.339950 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1818e16-95de-41d6-a2e9-40d58154404d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:11.352102 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5baa3-88d8-4af1-b562-6c96473a63c4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:11.380424 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754db9c-ca9c-4d24-924c-e75fc4c45024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:11.381108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754db9c-ca9c-4d24-924c-e75fc4c45024 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:11.381389 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754db9c-ca9c-4d24-924c-e75fc4c45024 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:11.381663 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754db9c-ca9c-4d24-924c-e75fc4c45024 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:11.382417 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754db9c-ca9c-4d24-924c-e75fc4c45024 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:11.382714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754db9c-ca9c-4d24-924c-e75fc4c45024 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:11.384257 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754db9c-ca9c-4d24-924c-e75fc4c45024 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:11.386746 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5eb72-1b1d-4609-916a-6155a513ce4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:11.387362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5eb72-1b1d-4609-916a-6155a513ce4d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:11.387641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5eb72-1b1d-4609-916a-6155a513ce4d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:11.387924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5eb72-1b1d-4609-916a-6155a513ce4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:11.388629 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5eb72-1b1d-4609-916a-6155a513ce4d None admin] RBAC: Adding query filter params (interface=public, service_id=0a7d1574b213469c917b749d536e104e, region_id=RegionOne) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:11.389021 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5eb72-1b1d-4609-916a-6155a513ce4d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:11.390418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5eb72-1b1d-4609-916a-6155a513ce4d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:11.397967 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 517/1033] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Sep 14 11:47:11 2022] GET /identity/v3/endpoints?service_id=0a7d1574b213469c917b749d536e104e&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:11.400930 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5eb72-1b1d-4609-916a-6155a513ce4d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:11.408873 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 517/1034] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Sep 14 11:47:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:11.410367 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754db9c-ca9c-4d24-924c-e75fc4c45024 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:11.447637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:11.465129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:11.514080 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5eb72-1b1d-4609-916a-6155a513ce4d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:11.518956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754db9c-ca9c-4d24-924c-e75fc4c45024 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:11.552625 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6e846-9065-478b-b939-20e65ba27d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:11.553242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6e846-9065-478b-b939-20e65ba27d4d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:11.553549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6e846-9065-478b-b939-20e65ba27d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:11.553792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6e846-9065-478b-b939-20e65ba27d4d None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:11.554427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6e846-9065-478b-b939-20e65ba27d4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:11.555791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6e846-9065-478b-b939-20e65ba27d4d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:11.556193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b658f79-8974-48a2-af54-d05c12d05089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:11.556803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b658f79-8974-48a2-af54-d05c12d05089 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:11.557085 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b658f79-8974-48a2-af54-d05c12d05089 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:11.557362 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b658f79-8974-48a2-af54-d05c12d05089 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:11.562660 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 518/1035] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Sep 14 11:47:11 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 11:47:11.634936 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b658f79-8974-48a2-af54-d05c12d05089 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:11.636469 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b658f79-8974-48a2-af54-d05c12d05089 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:11.689717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:11.748861 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 518/1036] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:47:11 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 14 11:47:11.754811 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6e846-9065-478b-b939-20e65ba27d4d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:11.871399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6e846-9065-478b-b939-20e65ba27d4d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:11.872924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:11.906827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca7c21-280e-4cec-964e-366baa31942d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:11.907458 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca7c21-280e-4cec-964e-366baa31942d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:11.907737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca7c21-280e-4cec-964e-366baa31942d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:11.908010 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca7c21-280e-4cec-964e-366baa31942d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:11.919048 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca7c21-280e-4cec-964e-366baa31942d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:11.920491 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca7c21-280e-4cec-964e-366baa31942d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:11.936168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-84ca7c21-280e-4cec-964e-366baa31942d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:47:11.936168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:11.936168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:11.936168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:11.936168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:11.936168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:11.936168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:11.936168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:11.936168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:11.936168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:11.936168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:11.936168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:11.936168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:47:11.936168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:47:11.936168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:47:11.936168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:47:11.937179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:11.937179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:11.937179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:11.937179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:11.937179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:11.937179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:11.937179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:11.937179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:11.937179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:11.937179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:11.937179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:11.937179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:11.937179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:11.937179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:11.937179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:11.937179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:11.937179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:47:11.945892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:47:11.945892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:47:11.945892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:11.945892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 519/1037] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:47:11.946168 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b658f79-8974-48a2-af54-d05c12d05089 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:12.051235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b658f79-8974-48a2-af54-d05c12d05089 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:12.087174 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16725f-cd12-4237-8cb2-2fb05e1b681e None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:12.087521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16725f-cd12-4237-8cb2-2fb05e1b681e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:12.087624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb9138-cc20-4e95-9de9-53fb3b030f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:12.087723 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16725f-cd12-4237-8cb2-2fb05e1b681e None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:12.088226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb9138-cc20-4e95-9de9-53fb3b030f6a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:12.088509 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb9138-cc20-4e95-9de9-53fb3b030f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:12.089313 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb9138-cc20-4e95-9de9-53fb3b030f6a None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:12.093354 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 520/1038] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:47:12.093479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb9138-cc20-4e95-9de9-53fb3b030f6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:12.093479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb9138-cc20-4e95-9de9-53fb3b030f6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:12.093479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb9138-cc20-4e95-9de9-53fb3b030f6a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:12.095721 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ae4be-ec08-4cde-b72a-bbcc6a335622 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:12.096003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ae4be-ec08-4cde-b72a-bbcc6a335622 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:12.096276 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ae4be-ec08-4cde-b72a-bbcc6a335622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:12.106662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 519/1039] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:12.110121 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb9138-cc20-4e95-9de9-53fb3b030f6a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:12.118233 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c43ae4be-ec08-4cde-b72a-bbcc6a335622 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:12.119827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c43ae4be-ec08-4cde-b72a-bbcc6a335622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:12.120882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:12.190217 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 521/1040] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 11:47:12.216452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb9138-cc20-4e95-9de9-53fb3b030f6a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:12.250626 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f02667-7e17-4ab7-a701-be23bf537076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:12.251262 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f02667-7e17-4ab7-a701-be23bf537076 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:12.251546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f02667-7e17-4ab7-a701-be23bf537076 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:12.251822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f02667-7e17-4ab7-a701-be23bf537076 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:12.258598 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f02667-7e17-4ab7-a701-be23bf537076 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:12.260035 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f02667-7e17-4ab7-a701-be23bf537076 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:12.271334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-59f02667-7e17-4ab7-a701-be23bf537076 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:47:12.271334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:12.271334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:12.271334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:12.271334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:12.271334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:12.271334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:12.271334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:12.271334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:12.271334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:12.271334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:12.271334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:12.271334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:47:12.271334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:47:12.271334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:47:12.272237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:47:12.272237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:12.272237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:12.272237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:12.272237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:12.272237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:12.272237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:12.272237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:12.272237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:12.272237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:12.272237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:12.272237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:12.272237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:12.272237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:12.272237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:12.272237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:12.272237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:12.278877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:47:12.278877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:47:12.278877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:47:12.278877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:47:12.278877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:47:12.278877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:47:12.278877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:47:12.278877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:12.278877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 520/1041] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:47:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:47:12.280891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ae4be-ec08-4cde-b72a-bbcc6a335622 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:12.368683 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:12.384412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ae4be-ec08-4cde-b72a-bbcc6a335622 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:12.419296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84efda4-4ef3-4b9f-9d4d-aae4cd8a04e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:12.419935 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84efda4-4ef3-4b9f-9d4d-aae4cd8a04e7 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:12.420175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84efda4-4ef3-4b9f-9d4d-aae4cd8a04e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:12.420454 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84efda4-4ef3-4b9f-9d4d-aae4cd8a04e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:12.421148 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84efda4-4ef3-4b9f-9d4d-aae4cd8a04e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:12.421458 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84efda4-4ef3-4b9f-9d4d-aae4cd8a04e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:12.422927 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84efda4-4ef3-4b9f-9d4d-aae4cd8a04e7 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:12.437963 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 522/1042] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:47:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:12.441555 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f02667-7e17-4ab7-a701-be23bf537076 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:12.459743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84efda4-4ef3-4b9f-9d4d-aae4cd8a04e7 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:12.515402 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:12.552098 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f02667-7e17-4ab7-a701-be23bf537076 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:12.560941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84efda4-4ef3-4b9f-9d4d-aae4cd8a04e7 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:12.587208 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27b467-636d-4898-8c62-478bdf878d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:12.587809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27b467-636d-4898-8c62-478bdf878d11 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:12.588091 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27b467-636d-4898-8c62-478bdf878d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:12.588365 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27b467-636d-4898-8c62-478bdf878d11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:12.589631 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27b467-636d-4898-8c62-478bdf878d11 None admin] RBAC: Adding query filter params (role.id=09d8286638fb49d8a4499662066fda1a, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=38b093bc4c1d4ebcbd2a6bb16b70c029) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:12.589968 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27b467-636d-4898-8c62-478bdf878d11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:12.591411 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27b467-636d-4898-8c62-478bdf878d11 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:12.596525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f8f0e-f9ab-42a1-890d-34528862306b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:12.597051 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 521/1043] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:47:12 2022] GET /identity/v3/role_assignments?user.id=38b093bc4c1d4ebcbd2a6bb16b70c029&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=09d8286638fb49d8a4499662066fda1a => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:12.597159 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f8f0e-f9ab-42a1-890d-34528862306b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:12.597419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f8f0e-f9ab-42a1-890d-34528862306b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:12.597693 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f8f0e-f9ab-42a1-890d-34528862306b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:12.598909 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f8f0e-f9ab-42a1-890d-34528862306b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:12.600358 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f8f0e-f9ab-42a1-890d-34528862306b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:12.605952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 523/1044] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:12.608162 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27b467-636d-4898-8c62-478bdf878d11 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:12.715021 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27b467-636d-4898-8c62-478bdf878d11 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:12.746164 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:12.751830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b78fb-35c7-496f-a6a4-6ee15cf5dd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:12.752452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b78fb-35c7-496f-a6a4-6ee15cf5dd4d None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:12.752729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b78fb-35c7-496f-a6a4-6ee15cf5dd4d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:12.753000 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b78fb-35c7-496f-a6a4-6ee15cf5dd4d None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:12.753677 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b78fb-35c7-496f-a6a4-6ee15cf5dd4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:12.755242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b78fb-35c7-496f-a6a4-6ee15cf5dd4d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:12.833887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 522/1045] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 14 11:47:12 2022] POST /identity/v3/users => generated 285 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:47:13.731934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fd973-635d-4aa6-8b0d-b264540de2cb None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:13.732460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fd973-635d-4aa6-8b0d-b264540de2cb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:13.732460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fd973-635d-4aa6-8b0d-b264540de2cb None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:13.734032 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 524/1046] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:47:13.738982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219bd0a-e136-4f6c-b110-bdd588703ce4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:13.739369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219bd0a-e136-4f6c-b110-bdd588703ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:13.739530 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219bd0a-e136-4f6c-b110-bdd588703ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:13.759694 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0219bd0a-e136-4f6c-b110-bdd588703ce4 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:13.760958 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0219bd0a-e136-4f6c-b110-bdd588703ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:13.837325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 523/1047] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:14.103783 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fd973-635d-4aa6-8b0d-b264540de2cb None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:14.119269 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:14.207564 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fd973-635d-4aa6-8b0d-b264540de2cb None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:14.240693 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e344e3a-0155-4d64-84d8-fa796b2a280e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:14.241313 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e344e3a-0155-4d64-84d8-fa796b2a280e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:14.241591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e344e3a-0155-4d64-84d8-fa796b2a280e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:14.241878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e344e3a-0155-4d64-84d8-fa796b2a280e None admin] PATH_INFO: `/v3/services/image` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:14.242462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e344e3a-0155-4d64-84d8-fa796b2a280e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:14.243816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e344e3a-0155-4d64-84d8-fa796b2a280e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:14.252923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-4e344e3a-0155-4d64-84d8-fa796b2a280e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 11:47:14.252923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:14.252923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:47:14.252923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:47:14.252923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:47:14.252923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:47:14.252923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:47:14.252923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:47:14.252923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 11:47:14.252923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:14.252923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:47:14.252923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:14.252923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:14.252923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:14.252923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:14.252923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:14.252923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:14.254177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:14.254177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:14.254177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:14.254177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:14.254177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:14.254177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:14.254177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:47:14.254177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:47:14.254177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:47:14.254177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:47:14.254177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:14.254177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:14.254177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:14.254177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:14.254177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:14.254177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:14.255068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:14.255068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:14.255068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:14.255068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:14.255068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:14.255068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:14.255068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:14.255068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:14.255068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:14.255068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:14.255068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:47:14.255068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:47:14.255068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 11:47:14.255068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:14.258191 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 525/1048] 10.222.0.62 () {56 vars in 1105 bytes} [Wed Sep 14 11:47:14 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:14.259944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219bd0a-e136-4f6c-b110-bdd588703ce4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:14.364940 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219bd0a-e136-4f6c-b110-bdd588703ce4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:14.382259 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:14.399540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363f4d7-182f-40b2-984f-89ecc28a3062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:14.400143 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363f4d7-182f-40b2-984f-89ecc28a3062 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:14.400490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363f4d7-182f-40b2-984f-89ecc28a3062 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:14.400694 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363f4d7-182f-40b2-984f-89ecc28a3062 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:14.401342 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363f4d7-182f-40b2-984f-89ecc28a3062 None admin] RBAC: Adding query filter params (name=image) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:14.401642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363f4d7-182f-40b2-984f-89ecc28a3062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:14.403027 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363f4d7-182f-40b2-984f-89ecc28a3062 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:14.408233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 524/1049] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:14.413507 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e344e3a-0155-4d64-84d8-fa796b2a280e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:14.491544 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:14.515314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e344e3a-0155-4d64-84d8-fa796b2a280e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:14.550908 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b3b5f-338d-440a-b37f-9bbbeb6ea24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:14.551556 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b3b5f-338d-440a-b37f-9bbbeb6ea24a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:14.551795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b3b5f-338d-440a-b37f-9bbbeb6ea24a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:14.552068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b3b5f-338d-440a-b37f-9bbbeb6ea24a None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:14.552697 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b3b5f-338d-440a-b37f-9bbbeb6ea24a None admin] RBAC: Adding query filter params (type=image) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:14.552999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b3b5f-338d-440a-b37f-9bbbeb6ea24a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:14.554397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b3b5f-338d-440a-b37f-9bbbeb6ea24a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:14.566199 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 526/1050] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:14.566358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363f4d7-182f-40b2-984f-89ecc28a3062 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:14.669508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363f4d7-182f-40b2-984f-89ecc28a3062 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:14.704233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c6b6f-a500-4cdf-a0b3-082654077603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:14.704840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c6b6f-a500-4cdf-a0b3-082654077603 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:14.705116 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c6b6f-a500-4cdf-a0b3-082654077603 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:14.705386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c6b6f-a500-4cdf-a0b3-082654077603 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:14.706085 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c6b6f-a500-4cdf-a0b3-082654077603 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:14.707430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c6b6f-a500-4cdf-a0b3-082654077603 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:14.763133 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787e90b-0b75-443f-9e57-b45f3171ed0d None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:14.763656 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787e90b-0b75-443f-9e57-b45f3171ed0d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:14.763751 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787e90b-0b75-443f-9e57-b45f3171ed0d None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:14.765029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 527/1051] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:14.769806 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b0e04-e6cf-439f-a878-c0e1a2d8c6dc None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:14.770121 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b0e04-e6cf-439f-a878-c0e1a2d8c6dc None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:14.770399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b0e04-e6cf-439f-a878-c0e1a2d8c6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:14.793883 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 525/1052] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 11:47:14 2022] POST /identity/v3/endpoints => generated 327 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 11:47:14.795729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-981b0e04-e6cf-439f-a878-c0e1a2d8c6dc None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:14.797027 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-981b0e04-e6cf-439f-a878-c0e1a2d8c6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:14.889052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 528/1053] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 11:47:15.043993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb06a95-02ab-40c1-a618-bfee22444003 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:15.044291 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb06a95-02ab-40c1-a618-bfee22444003 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:15.044462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb06a95-02ab-40c1-a618-bfee22444003 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:15.045990 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 526/1054] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:15.051430 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0baa036-3783-49a3-90be-04a5a40ac1d9 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:15.051720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0baa036-3783-49a3-90be-04a5a40ac1d9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:15.051995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0baa036-3783-49a3-90be-04a5a40ac1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:15.070188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c0baa036-3783-49a3-90be-04a5a40ac1d9 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:15.071377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c0baa036-3783-49a3-90be-04a5a40ac1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:15.131446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:15.139166 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 529/1055] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:15.167811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb06a95-02ab-40c1-a618-bfee22444003 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:15.211391 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:15.272179 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb06a95-02ab-40c1-a618-bfee22444003 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:15.307936 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3545bff-77f1-4be0-bddf-2ff0a6fa7405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:15.308487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3545bff-77f1-4be0-bddf-2ff0a6fa7405 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:15.308779 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3545bff-77f1-4be0-bddf-2ff0a6fa7405 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:15.309064 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3545bff-77f1-4be0-bddf-2ff0a6fa7405 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:15.320153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3545bff-77f1-4be0-bddf-2ff0a6fa7405 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:47:15.325514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-e3545bff-77f1-4be0-bddf-2ff0a6fa7405 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 11:47:15.325514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:15.325514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:15.325514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:15.325514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:15.325514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:15.325514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:15.325514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:15.325514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:15.325514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:15.325514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:15.325514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:15.325514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:47:15.325514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:47:15.325514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:47:15.325514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:47:15.326482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:47:15.326482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:47:15.326482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:47:15.326482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:47:15.326482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:15.326482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:15.326482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:15.326482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:15.326482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:15.326482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:15.326482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:15.326482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:15.326482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:15.326482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:15.326482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:15.326482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:15.327366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:15.327366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:15.327366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:15.327366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:15.327366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:47:15.327366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:47:15.327366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:47:15.327366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:47:15.327366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:47:15.327366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:47:15.327366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 11:47:15.327366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:15.327366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 527/1056] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Sep 14 11:47:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:15.332725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0baa036-3783-49a3-90be-04a5a40ac1d9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:15.431933 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3545bff-77f1-4be0-bddf-2ff0a6fa7405 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:15.434384 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0baa036-3783-49a3-90be-04a5a40ac1d9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:15.469839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976fbcd-fe7b-4afa-b59d-25528b24e040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:15.470477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976fbcd-fe7b-4afa-b59d-25528b24e040 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:15.470766 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976fbcd-fe7b-4afa-b59d-25528b24e040 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:15.471041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976fbcd-fe7b-4afa-b59d-25528b24e040 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:15.471708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976fbcd-fe7b-4afa-b59d-25528b24e040 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:15.472055 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976fbcd-fe7b-4afa-b59d-25528b24e040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:15.473454 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976fbcd-fe7b-4afa-b59d-25528b24e040 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:15.481918 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 530/1057] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Sep 14 11:47:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:15.487354 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976fbcd-fe7b-4afa-b59d-25528b24e040 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:15.540290 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:15.544274 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3545bff-77f1-4be0-bddf-2ff0a6fa7405 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:15.548983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:15.581004 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006944b6-1d35-448d-987e-e3801e1b47fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:15.581558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006944b6-1d35-448d-987e-e3801e1b47fe None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:15.581845 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006944b6-1d35-448d-987e-e3801e1b47fe None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:15.582162 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006944b6-1d35-448d-987e-e3801e1b47fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:15.593138 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006944b6-1d35-448d-987e-e3801e1b47fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:47:15.594294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976fbcd-fe7b-4afa-b59d-25528b24e040 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:15.597962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-006944b6-1d35-448d-987e-e3801e1b47fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:47:15.597962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:15.597962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:15.597962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:15.597962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:15.597962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:15.597962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:15.597962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:15.597962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:15.597962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:15.597962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:15.597962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:15.597962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:47:15.597962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:47:15.597962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:47:15.597962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:47:15.598888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:47:15.598888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:47:15.598888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:47:15.598888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:47:15.598888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:15.598888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:15.598888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:15.598888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:15.598888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:15.598888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:15.598888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:15.598888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:15.598888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:15.598888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:15.598888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:15.598888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:15.599739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:15.599739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:15.599739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:15.599739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:15.599739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:47:15.599739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:47:15.599739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:47:15.599739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:47:15.599739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:47:15.599739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:47:15.599739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:47:15.599739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:15.604352 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 528/1058] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:47:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:47:15.605060 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006944b6-1d35-448d-987e-e3801e1b47fe None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:15.629245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f646be2-f313-4d6b-a62c-352c2d5e3646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:15.629912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f646be2-f313-4d6b-a62c-352c2d5e3646 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:15.630189 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f646be2-f313-4d6b-a62c-352c2d5e3646 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:15.630467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f646be2-f313-4d6b-a62c-352c2d5e3646 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:15.642296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f646be2-f313-4d6b-a62c-352c2d5e3646 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:15.643750 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f646be2-f313-4d6b-a62c-352c2d5e3646 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:15.661181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-9f646be2-f313-4d6b-a62c-352c2d5e3646 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:47:15.661181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:15.661181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:15.661181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:15.661181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:15.661181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:15.661181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:15.661181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:15.661181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:15.661181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:15.661181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:15.661181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:15.661181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:47:15.661181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:47:15.661181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:47:15.661181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:47:15.662259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:15.662259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:15.662259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:15.662259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:15.662259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:15.662259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:15.662259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:15.662259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:15.662259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:15.662259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:15.662259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:15.662259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:15.662259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:15.662259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:15.662259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:15.662259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:15.662259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:47:15.663165 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:47:15.663165 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:47:15.663165 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:15.663165 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 531/1059] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:47:15.667861 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f646be2-f313-4d6b-a62c-352c2d5e3646 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:15.719848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006944b6-1d35-448d-987e-e3801e1b47fe None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:15.756104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290dda4-e7c7-4136-91c1-441a760ce0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:15.756736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290dda4-e7c7-4136-91c1-441a760ce0de None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:15.757011 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290dda4-e7c7-4136-91c1-441a760ce0de None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:15.757292 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290dda4-e7c7-4136-91c1-441a760ce0de None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:15.757962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290dda4-e7c7-4136-91c1-441a760ce0de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:15.758276 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290dda4-e7c7-4136-91c1-441a760ce0de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:15.759765 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290dda4-e7c7-4136-91c1-441a760ce0de None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:15.774005 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 529/1060] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:47:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:15.775514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290dda4-e7c7-4136-91c1-441a760ce0de None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:15.778591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f646be2-f313-4d6b-a62c-352c2d5e3646 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:15.790804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:15.798676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:15.815923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43eeb4-f94b-40a6-93e2-4ba39464cf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:15.816526 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43eeb4-f94b-40a6-93e2-4ba39464cf68 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:15.817030 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43eeb4-f94b-40a6-93e2-4ba39464cf68 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:15.817149 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43eeb4-f94b-40a6-93e2-4ba39464cf68 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:15.817743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43eeb4-f94b-40a6-93e2-4ba39464cf68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:15.818081 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43eeb4-f94b-40a6-93e2-4ba39464cf68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:15.820991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43eeb4-f94b-40a6-93e2-4ba39464cf68 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:15.835021 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 532/1061] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:15.840448 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43eeb4-f94b-40a6-93e2-4ba39464cf68 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:15.890262 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290dda4-e7c7-4136-91c1-441a760ce0de None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:15.927735 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1517782-1c61-42a8-8fd7-b10c6911d67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:15.928363 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1517782-1c61-42a8-8fd7-b10c6911d67b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:15.928902 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1517782-1c61-42a8-8fd7-b10c6911d67b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:15.928902 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1517782-1c61-42a8-8fd7-b10c6911d67b None admin] PATH_INFO: `/v3/users/623b5af81ef34c8cb7450151fb883146` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:15.941125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43eeb4-f94b-40a6-93e2-4ba39464cf68 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:15.942963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1517782-1c61-42a8-8fd7-b10c6911d67b None admin] RBAC: Authorizing `identity:get_user(user_id=623b5af81ef34c8cb7450151fb883146)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:15.944385 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1517782-1c61-42a8-8fd7-b10c6911d67b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:15.946196 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 530/1062] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:47:15 2022] GET /identity/v3/users/623b5af81ef34c8cb7450151fb883146 => generated 285 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:15.951728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1517782-1c61-42a8-8fd7-b10c6911d67b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:15.985988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edbfa9-3bc7-42bc-b5d5-5632857e84f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:15.986625 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edbfa9-3bc7-42bc-b5d5-5632857e84f1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:15.986902 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edbfa9-3bc7-42bc-b5d5-5632857e84f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:15.987174 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edbfa9-3bc7-42bc-b5d5-5632857e84f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:15.994222 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58edbfa9-3bc7-42bc-b5d5-5632857e84f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:15.996172 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58edbfa9-3bc7-42bc-b5d5-5632857e84f1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:16.008035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-58edbfa9-3bc7-42bc-b5d5-5632857e84f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 11:47:16.008035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:16.008035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:16.008035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:16.008035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:16.008035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:16.008035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:16.008035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:16.008035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:16.008035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:16.008035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:16.008035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:16.008035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:47:16.008035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:47:16.008035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:47:16.008035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:47:16.008977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:16.008977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:16.008977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:47:16.008977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:47:16.008977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:47:16.008977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:47:16.008977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:16.008977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:16.008977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:16.008977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:16.008977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:16.008977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:16.008977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:16.008977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:16.008977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:16.008977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:16.008977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:16.008977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:16.009994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:16.009994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:16.009994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:47:16.009994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:47:16.009994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:47:16.009994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:47:16.009994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:47:16.009994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:47:16.009994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 11:47:16.009994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:16.009994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 533/1063] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Sep 14 11:47:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:47:16.014826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edbfa9-3bc7-42bc-b5d5-5632857e84f1 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:16.036350 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:16.048788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:16.058809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1517782-1c61-42a8-8fd7-b10c6911d67b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:16.095537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a91f98-cab6-4c93-b51d-8c3ad60d651a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:16.096142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a91f98-cab6-4c93-b51d-8c3ad60d651a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:16.096433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a91f98-cab6-4c93-b51d-8c3ad60d651a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:16.096710 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a91f98-cab6-4c93-b51d-8c3ad60d651a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:16.121187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edbfa9-3bc7-42bc-b5d5-5632857e84f1 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:16.154428 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda325f-978e-4163-ab75-05727052c572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:16.155058 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda325f-978e-4163-ab75-05727052c572 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:16.155324 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda325f-978e-4163-ab75-05727052c572 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:16.155691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda325f-978e-4163-ab75-05727052c572 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:16.156338 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda325f-978e-4163-ab75-05727052c572 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:16.156651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda325f-978e-4163-ab75-05727052c572 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:16.158134 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda325f-978e-4163-ab75-05727052c572 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:16.167946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a91f98-cab6-4c93-b51d-8c3ad60d651a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:16.169631 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a91f98-cab6-4c93-b51d-8c3ad60d651a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:16.176627 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 534/1064] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Sep 14 11:47:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:16.183230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda325f-978e-4163-ab75-05727052c572 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:16.258872 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:16.275063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:16.280960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 531/1065] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:47:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 14 11:47:16.286851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda325f-978e-4163-ab75-05727052c572 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:16.287001 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a91f98-cab6-4c93-b51d-8c3ad60d651a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:16.321843 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7dd65-5e63-4787-a8c5-fa4c3a79bc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:16.322691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7dd65-5e63-4787-a8c5-fa4c3a79bc90 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:16.324042 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7dd65-5e63-4787-a8c5-fa4c3a79bc90 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:16.324323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7dd65-5e63-4787-a8c5-fa4c3a79bc90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:16.335143 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff7dd65-5e63-4787-a8c5-fa4c3a79bc90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:16.337062 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff7dd65-5e63-4787-a8c5-fa4c3a79bc90 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:16.354699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-1ff7dd65-5e63-4787-a8c5-fa4c3a79bc90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:47:16.354699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:16.354699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:16.354699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:16.354699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:16.354699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:16.354699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:16.354699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:16.354699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:16.354699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:16.354699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:16.354699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:16.354699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:47:16.354699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:47:16.354699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:47:16.354699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:47:16.355637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:16.355637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:16.355637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:16.355637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:16.355637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:16.355637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:16.355637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:16.355637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:16.355637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:16.355637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:16.355637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:16.355637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:16.355637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:16.355637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:16.355637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:16.355637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:16.355637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:47:16.356559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:47:16.356559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:47:16.356559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:16.356559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 535/1066] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:47:16.361573 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7dd65-5e63-4787-a8c5-fa4c3a79bc90 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:16.391769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a91f98-cab6-4c93-b51d-8c3ad60d651a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:16.426850 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690de074-39a3-4c6b-9876-7b7cfef93d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:16.427482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690de074-39a3-4c6b-9876-7b7cfef93d7e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:16.427761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690de074-39a3-4c6b-9876-7b7cfef93d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:16.428037 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690de074-39a3-4c6b-9876-7b7cfef93d7e None admin] PATH_INFO: `/v3/projects/3c8ee4546b6c44c78943e2cb9864ec15` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:16.429151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690de074-39a3-4c6b-9876-7b7cfef93d7e None admin] RBAC: Authorizing `identity:get_project(project_id=3c8ee4546b6c44c78943e2cb9864ec15)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:16.430612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690de074-39a3-4c6b-9876-7b7cfef93d7e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:16.432239 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 532/1067] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:47:16 2022] GET /identity/v3/projects/3c8ee4546b6c44c78943e2cb9864ec15 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:16.438379 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690de074-39a3-4c6b-9876-7b7cfef93d7e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:16.469478 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7dd65-5e63-4787-a8c5-fa4c3a79bc90 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:16.485334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:16.505974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5acbf6-42be-4572-afc5-9e45369a1b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:16.506590 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5acbf6-42be-4572-afc5-9e45369a1b24 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:16.506873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5acbf6-42be-4572-afc5-9e45369a1b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:16.507150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5acbf6-42be-4572-afc5-9e45369a1b24 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:16.507801 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5acbf6-42be-4572-afc5-9e45369a1b24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:16.508106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5acbf6-42be-4572-afc5-9e45369a1b24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:16.509502 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5acbf6-42be-4572-afc5-9e45369a1b24 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:16.518655 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:16.527744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 536/1068] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:16.528681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5acbf6-42be-4572-afc5-9e45369a1b24 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:16.542953 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690de074-39a3-4c6b-9876-7b7cfef93d7e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:16.577720 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8d3de-8157-4a05-89eb-b9fd3eb0c35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:16.578365 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8d3de-8157-4a05-89eb-b9fd3eb0c35b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:16.578656 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8d3de-8157-4a05-89eb-b9fd3eb0c35b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:16.578936 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8d3de-8157-4a05-89eb-b9fd3eb0c35b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:16.579659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8d3de-8157-4a05-89eb-b9fd3eb0c35b None admin] RBAC: Adding query filter params (role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.project.id=3c8ee4546b6c44c78943e2cb9864ec15, user.id=623b5af81ef34c8cb7450151fb883146) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:16.579973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8d3de-8157-4a05-89eb-b9fd3eb0c35b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:16.581405 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8d3de-8157-4a05-89eb-b9fd3eb0c35b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:16.586815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 533/1069] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:47:16 2022] GET /identity/v3/role_assignments?user.id=623b5af81ef34c8cb7450151fb883146&scope.project.id=3c8ee4546b6c44c78943e2cb9864ec15&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:16.647884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5acbf6-42be-4572-afc5-9e45369a1b24 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:16.686458 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30735563-792a-461d-b4a0-bd1b9230d04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:16.687104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30735563-792a-461d-b4a0-bd1b9230d04e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:16.687389 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30735563-792a-461d-b4a0-bd1b9230d04e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:16.687668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30735563-792a-461d-b4a0-bd1b9230d04e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:16.695070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30735563-792a-461d-b4a0-bd1b9230d04e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:16.696546 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30735563-792a-461d-b4a0-bd1b9230d04e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:16.708768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-30735563-792a-461d-b4a0-bd1b9230d04e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:47:16.708768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:16.708768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:16.708768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:16.708768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:16.708768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:16.708768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:16.708768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:16.708768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:16.708768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:16.708768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:16.708768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:16.708768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:47:16.708768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:47:16.708768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:47:16.709788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:47:16.709788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:16.709788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:16.709788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:16.709788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:16.709788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:16.709788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:16.709788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:16.709788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:16.709788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:16.709788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:16.709788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:16.709788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:16.709788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:16.709788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:16.709788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:16.709788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:16.711318 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:47:16.711318 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:47:16.711318 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:47:16.711318 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:47:16.711318 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:47:16.711318 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:47:16.711318 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:47:16.711318 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:16.715352 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 537/1070] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:47:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:16.716379 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8d3de-8157-4a05-89eb-b9fd3eb0c35b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:16.833350 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8d3de-8157-4a05-89eb-b9fd3eb0c35b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:16.845265 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:16.868091 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54badc-038c-4ea7-97e6-551b625f7fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:16.868702 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54badc-038c-4ea7-97e6-551b625f7fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:16.869056 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54badc-038c-4ea7-97e6-551b625f7fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:16.869339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54badc-038c-4ea7-97e6-551b625f7fc9 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:16.870057 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54badc-038c-4ea7-97e6-551b625f7fc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:16.870368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54badc-038c-4ea7-97e6-551b625f7fc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:16.871788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54badc-038c-4ea7-97e6-551b625f7fc9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:16.890675 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 534/1071] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:47:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:16.892988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30735563-792a-461d-b4a0-bd1b9230d04e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:16.956647 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:17.000282 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30735563-792a-461d-b4a0-bd1b9230d04e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:17.004601 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c38945-4a3b-419d-aeb6-b13007186c9b None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:17.004899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c38945-4a3b-419d-aeb6-b13007186c9b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:17.005180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c38945-4a3b-419d-aeb6-b13007186c9b None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:17.006716 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 535/1072] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:47:17.012000 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e28c37-a418-45a1-80aa-9659f890f4b8 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:17.012284 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e28c37-a418-45a1-80aa-9659f890f4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:17.012552 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e28c37-a418-45a1-80aa-9659f890f4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:17.030663 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e8e28c37-a418-45a1-80aa-9659f890f4b8 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:17.031858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e8e28c37-a418-45a1-80aa-9659f890f4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:17.037425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c6057-1cd1-42fd-9d50-d2f57f952ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:17.038329 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c6057-1cd1-42fd-9d50-d2f57f952ae9 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:17.038786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c6057-1cd1-42fd-9d50-d2f57f952ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:17.039272 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c6057-1cd1-42fd-9d50-d2f57f952ae9 None admin] PATH_INFO: `/v3/projects/449a5fac1acd478abea544446c08ecd9/users/38b093bc4c1d4ebcbd2a6bb16b70c029/roles/09d8286638fb49d8a4499662066fda1a` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:17.041574 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c6057-1cd1-42fd-9d50-d2f57f952ae9 None admin] RBAC: Authorizing `identity:create_grant(project_id=449a5fac1acd478abea544446c08ecd9, user_id=38b093bc4c1d4ebcbd2a6bb16b70c029, role_id=09d8286638fb49d8a4499662066fda1a)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:17.043222 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c6057-1cd1-42fd-9d50-d2f57f952ae9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:17.099261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 536/1073] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 11:47:17.106835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 538/1074] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:47:16 2022] PUT /identity/v3/projects/449a5fac1acd478abea544446c08ecd9/users/38b093bc4c1d4ebcbd2a6bb16b70c029/roles/09d8286638fb49d8a4499662066fda1a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 11:47:17.368406 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e28c37-a418-45a1-80aa-9659f890f4b8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:17.469112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:17.472804 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e28c37-a418-45a1-80aa-9659f890f4b8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:17.506756 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbe76f-3d8e-4ebd-852b-7c32c068d1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:17.507361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbe76f-3d8e-4ebd-852b-7c32c068d1ab None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:17.507640 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbe76f-3d8e-4ebd-852b-7c32c068d1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:17.507914 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbe76f-3d8e-4ebd-852b-7c32c068d1ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:17.577909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cbe76f-3d8e-4ebd-852b-7c32c068d1ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:17.577909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cbe76f-3d8e-4ebd-852b-7c32c068d1ab None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:17.672869 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:17.677895 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 537/1075] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:47:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:47:17.684010 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c6057-1cd1-42fd-9d50-d2f57f952ae9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:17.784397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c6057-1cd1-42fd-9d50-d2f57f952ae9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:17.805162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:17.823553 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ef73e-6559-4f3b-a980-7529160c3042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:17.823921 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ef73e-6559-4f3b-a980-7529160c3042 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:17.824213 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ef73e-6559-4f3b-a980-7529160c3042 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:17.824521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ef73e-6559-4f3b-a980-7529160c3042 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:17.836064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ef73e-6559-4f3b-a980-7529160c3042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:17.837531 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ef73e-6559-4f3b-a980-7529160c3042 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:17.853112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-676ef73e-6559-4f3b-a980-7529160c3042 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:47:17.853112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:17.853112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:17.853112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:17.853112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:17.853112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:17.853112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:17.853112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:17.853112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:17.853112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:17.853112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:17.853112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:17.853112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:47:17.853112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:47:17.853112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:47:17.853112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:47:17.854689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:17.854689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:17.854689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:17.854689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:17.854689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:17.854689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:17.854689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:17.854689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:17.854689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:17.854689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:17.854689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:17.854689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:17.854689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:17.854689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:17.854689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:17.854689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:17.854689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:47:17.855612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:47:17.855612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:47:17.855612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:17.855612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 539/1076] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:17.860789 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbe76f-3d8e-4ebd-852b-7c32c068d1ab None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:17.960534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbe76f-3d8e-4ebd-852b-7c32c068d1ab None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:17.995651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42d493-b580-4cc5-a1e2-cc13c1f52195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:17.996272 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42d493-b580-4cc5-a1e2-cc13c1f52195 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:17.996553 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42d493-b580-4cc5-a1e2-cc13c1f52195 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:17.996827 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42d493-b580-4cc5-a1e2-cc13c1f52195 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:17.997476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42d493-b580-4cc5-a1e2-cc13c1f52195 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:17.997778 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42d493-b580-4cc5-a1e2-cc13c1f52195 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:17.999165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42d493-b580-4cc5-a1e2-cc13c1f52195 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:18.017896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 538/1077] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:18.720403 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7658317-60b7-4cee-98af-6408c8cb4236 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:18.720950 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7658317-60b7-4cee-98af-6408c8cb4236 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:18.720950 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7658317-60b7-4cee-98af-6408c8cb4236 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:18.722544 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 540/1078] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:18.727501 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665969cb-7425-4fdd-8f51-304dc7938652 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:18.727793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665969cb-7425-4fdd-8f51-304dc7938652 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:18.728074 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665969cb-7425-4fdd-8f51-304dc7938652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:18.746364 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-665969cb-7425-4fdd-8f51-304dc7938652 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:18.747553 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-665969cb-7425-4fdd-8f51-304dc7938652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:18.801524 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:18.814132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 539/1079] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:19.094574 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7658317-60b7-4cee-98af-6408c8cb4236 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:19.201051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7658317-60b7-4cee-98af-6408c8cb4236 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:19.234314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8550f0-58da-4762-9fbd-e7c7b82b14d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:19.234944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8550f0-58da-4762-9fbd-e7c7b82b14d2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:19.235235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8550f0-58da-4762-9fbd-e7c7b82b14d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:19.235512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8550f0-58da-4762-9fbd-e7c7b82b14d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:19.246482 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8550f0-58da-4762-9fbd-e7c7b82b14d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:47:19.251253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-5f8550f0-58da-4762-9fbd-e7c7b82b14d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:47:19.251253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:19.251253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:19.251253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:19.251253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:19.251253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:19.251253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:19.251253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:19.251253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:19.251253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:19.251253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:19.251253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:19.251253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:47:19.251253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:47:19.251253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:47:19.251253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:47:19.252363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:47:19.252363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:47:19.252363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:47:19.252363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:47:19.252363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:19.252363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:19.252363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:19.252363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:19.252363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:19.252363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:19.252363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:19.252363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:19.252363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:19.252363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:19.252363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:19.252363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:19.253638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:19.253638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:19.253638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:19.253638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:19.253638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:47:19.253638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:47:19.253638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:47:19.253638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:47:19.253638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:47:19.253638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:47:19.253638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:47:19.253638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:19.253638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 541/1080] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:47:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:19.258351 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665969cb-7425-4fdd-8f51-304dc7938652 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:19.260152 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2aefe7-f3f2-4691-ad1b-fc7004094878 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:19.260486 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2aefe7-f3f2-4691-ad1b-fc7004094878 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:19.260766 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2aefe7-f3f2-4691-ad1b-fc7004094878 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:19.262101 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 542/1081] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:47:19.266858 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e11a6-346b-47be-bc9c-e1fca9384258 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:19.267142 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e11a6-346b-47be-bc9c-e1fca9384258 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:19.267417 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e11a6-346b-47be-bc9c-e1fca9384258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:19.284851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9b7e11a6-346b-47be-bc9c-e1fca9384258 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:19.286148 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9b7e11a6-346b-47be-bc9c-e1fca9384258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:19.312832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:19.359505 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 543/1082] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 11:47:19.360900 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665969cb-7425-4fdd-8f51-304dc7938652 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:19.394117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487de6d6-ba75-4298-aa95-eed8dc91d19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:19.394718 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487de6d6-ba75-4298-aa95-eed8dc91d19d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:19.395001 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487de6d6-ba75-4298-aa95-eed8dc91d19d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:19.395274 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487de6d6-ba75-4298-aa95-eed8dc91d19d None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:19.395929 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487de6d6-ba75-4298-aa95-eed8dc91d19d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:19.396231 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487de6d6-ba75-4298-aa95-eed8dc91d19d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:19.397586 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487de6d6-ba75-4298-aa95-eed8dc91d19d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:19.405395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 540/1083] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:47:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:19.411262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e11a6-346b-47be-bc9c-e1fca9384258 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:19.509962 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e11a6-346b-47be-bc9c-e1fca9384258 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:19.543464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372d33a-b182-4d8f-86a0-f650f669d6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:19.544088 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372d33a-b182-4d8f-86a0-f650f669d6f2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:19.544380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372d33a-b182-4d8f-86a0-f650f669d6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:19.544658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372d33a-b182-4d8f-86a0-f650f669d6f2 None admin] PATH_INFO: `/v3/users/623b5af81ef34c8cb7450151fb883146` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:19.545787 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372d33a-b182-4d8f-86a0-f650f669d6f2 None admin] RBAC: Authorizing `identity:get_user(user_id=623b5af81ef34c8cb7450151fb883146)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:19.547237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372d33a-b182-4d8f-86a0-f650f669d6f2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:19.549064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 544/1084] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:47:19 2022] GET /identity/v3/users/623b5af81ef34c8cb7450151fb883146 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:19.554766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487de6d6-ba75-4298-aa95-eed8dc91d19d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:19.606150 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:19.622162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372d33a-b182-4d8f-86a0-f650f669d6f2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:19.652274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:19.657547 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487de6d6-ba75-4298-aa95-eed8dc91d19d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:19.691029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8a9f9-b100-4560-bb0f-e6b9bfa323ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:19.691651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8a9f9-b100-4560-bb0f-e6b9bfa323ea None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:19.691927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8a9f9-b100-4560-bb0f-e6b9bfa323ea None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:19.692195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8a9f9-b100-4560-bb0f-e6b9bfa323ea None admin] PATH_INFO: `/v3/projects/3c8ee4546b6c44c78943e2cb9864ec15` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:19.693291 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8a9f9-b100-4560-bb0f-e6b9bfa323ea None admin] RBAC: Authorizing `identity:get_project(project_id=3c8ee4546b6c44c78943e2cb9864ec15)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:19.694732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8a9f9-b100-4560-bb0f-e6b9bfa323ea None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:19.696348 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 541/1085] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:47:19 2022] GET /identity/v3/projects/3c8ee4546b6c44c78943e2cb9864ec15 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:19.702232 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8a9f9-b100-4560-bb0f-e6b9bfa323ea None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:19.723685 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372d33a-b182-4d8f-86a0-f650f669d6f2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:19.757018 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3775bbd-e05d-4391-b15d-29c70331b12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:19.757631 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3775bbd-e05d-4391-b15d-29c70331b12c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:19.757943 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3775bbd-e05d-4391-b15d-29c70331b12c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:19.758220 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3775bbd-e05d-4391-b15d-29c70331b12c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:19.768829 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3775bbd-e05d-4391-b15d-29c70331b12c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:47:19.773643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-b3775bbd-e05d-4391-b15d-29c70331b12c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 11:47:19.773643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:19.773643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:19.773643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:19.773643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:19.773643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:19.773643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:19.773643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:19.773643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:19.773643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:19.773643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:19.773643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:19.773643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:47:19.773643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:47:19.773643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:47:19.773643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:47:19.774668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:47:19.774668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:47:19.774668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:47:19.774668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:47:19.774668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:19.774668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:19.774668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:19.774668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:19.774668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:19.774668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:19.774668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:19.774668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:19.774668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:19.774668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:19.774668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:19.774668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:19.775554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:19.775554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:19.775554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:19.775554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:19.775554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:47:19.775554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:47:19.775554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:47:19.775554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:47:19.775554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:47:19.775554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:47:19.775554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 11:47:19.775554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:19.775554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 545/1086] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Sep 14 11:47:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:47:19.781111 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3775bbd-e05d-4391-b15d-29c70331b12c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:19.802459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8a9f9-b100-4560-bb0f-e6b9bfa323ea None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:19.825685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:19.840315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cacded3-259e-4218-8c25-5920432e4c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:19.841129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cacded3-259e-4218-8c25-5920432e4c64 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:19.841527 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cacded3-259e-4218-8c25-5920432e4c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:19.841805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cacded3-259e-4218-8c25-5920432e4c64 None admin] PATH_INFO: `/v3/projects/3c8ee4546b6c44c78943e2cb9864ec15/users/623b5af81ef34c8cb7450151fb883146/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:19.843853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cacded3-259e-4218-8c25-5920432e4c64 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c8ee4546b6c44c78943e2cb9864ec15, user_id=623b5af81ef34c8cb7450151fb883146, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:19.845281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cacded3-259e-4218-8c25-5920432e4c64 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:19.871753 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:19.881260 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3775bbd-e05d-4391-b15d-29c70331b12c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:19.892801 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 542/1087] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:47:19 2022] PUT /identity/v3/projects/3c8ee4546b6c44c78943e2cb9864ec15/users/623b5af81ef34c8cb7450151fb883146/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 11:47:19.918096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dda9bb-7e5c-417b-81c8-51c1f500cf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:19.918733 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dda9bb-7e5c-417b-81c8-51c1f500cf48 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:19.919020 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dda9bb-7e5c-417b-81c8-51c1f500cf48 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:19.919327 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dda9bb-7e5c-417b-81c8-51c1f500cf48 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:19.920057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dda9bb-7e5c-417b-81c8-51c1f500cf48 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:19.920369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dda9bb-7e5c-417b-81c8-51c1f500cf48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:19.921810 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dda9bb-7e5c-417b-81c8-51c1f500cf48 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:19.931458 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 546/1088] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Sep 14 11:47:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:19.937557 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cacded3-259e-4218-8c25-5920432e4c64 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:20.043865 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cacded3-259e-4218-8c25-5920432e4c64 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:20.080124 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c528cb-46bd-4d8c-971a-da7dc62ecb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:20.080754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c528cb-46bd-4d8c-971a-da7dc62ecb0a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:20.081033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c528cb-46bd-4d8c-971a-da7dc62ecb0a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:20.081301 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c528cb-46bd-4d8c-971a-da7dc62ecb0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:20.092201 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c528cb-46bd-4d8c-971a-da7dc62ecb0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:20.093626 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c528cb-46bd-4d8c-971a-da7dc62ecb0a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:20.107186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-98c528cb-46bd-4d8c-971a-da7dc62ecb0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:47:20.107186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:20.107186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:20.107186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:20.107186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:20.107186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:20.107186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:20.107186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:20.107186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:20.107186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:20.107186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:20.107186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:20.107186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:47:20.107186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:47:20.107186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:47:20.107186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:47:20.108306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:20.108306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:20.108306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:20.108306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:20.108306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:20.108306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:20.108306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:20.108306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:20.108306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:20.108306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:20.108306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:20.108306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:20.108306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:20.108306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:20.108306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:20.108306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:20.108306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:47:20.109214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:47:20.109214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:47:20.109214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:20.109214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 543/1089] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:20.114015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dda9bb-7e5c-417b-81c8-51c1f500cf48 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:20.212825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dda9bb-7e5c-417b-81c8-51c1f500cf48 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:20.246337 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24558d46-3fb3-4803-9ef0-c44c4269f293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:20.246944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24558d46-3fb3-4803-9ef0-c44c4269f293 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:20.247227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24558d46-3fb3-4803-9ef0-c44c4269f293 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:20.247502 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24558d46-3fb3-4803-9ef0-c44c4269f293 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:20.248141 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24558d46-3fb3-4803-9ef0-c44c4269f293 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:20.248445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24558d46-3fb3-4803-9ef0-c44c4269f293 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:20.249808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24558d46-3fb3-4803-9ef0-c44c4269f293 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:20.262121 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 547/1090] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:20.267557 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c528cb-46bd-4d8c-971a-da7dc62ecb0a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:20.297762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:20.364490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c528cb-46bd-4d8c-971a-da7dc62ecb0a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:20.398713 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb201f2-d14a-43a8-8483-0db58fe638ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:20.399332 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb201f2-d14a-43a8-8483-0db58fe638ae None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:20.399609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb201f2-d14a-43a8-8483-0db58fe638ae None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:20.399872 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb201f2-d14a-43a8-8483-0db58fe638ae None admin] PATH_INFO: `/v3/users/nova` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:20.406826 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb201f2-d14a-43a8-8483-0db58fe638ae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:20.408264 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb201f2-d14a-43a8-8483-0db58fe638ae None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:20.419648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-ccb201f2-d14a-43a8-8483-0db58fe638ae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 11:47:20.419648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:20.419648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:20.419648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:20.419648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:20.419648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:20.419648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:20.419648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:20.419648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:20.419648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:20.419648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:20.419648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:20.419648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:47:20.419648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:47:20.419648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:47:20.419648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:47:20.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:20.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:20.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:47:20.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:47:20.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:47:20.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:47:20.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:20.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:20.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:20.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:20.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:20.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:20.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:20.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:20.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:20.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:20.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:20.420602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:20.421554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:20.421554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:20.421554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:47:20.421554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:47:20.421554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:47:20.421554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:47:20.421554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:47:20.421554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:47:20.421554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 11:47:20.421554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:20.421554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 544/1091] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Sep 14 11:47:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:20.426813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24558d46-3fb3-4803-9ef0-c44c4269f293 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:20.441838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:20.527655 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24558d46-3fb3-4803-9ef0-c44c4269f293 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:20.560675 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41474efa-9ff0-470c-95f6-fa0ec357393a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:20.561272 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41474efa-9ff0-470c-95f6-fa0ec357393a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:20.561554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41474efa-9ff0-470c-95f6-fa0ec357393a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:20.561823 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41474efa-9ff0-470c-95f6-fa0ec357393a None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:20.562569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41474efa-9ff0-470c-95f6-fa0ec357393a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:20.562874 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41474efa-9ff0-470c-95f6-fa0ec357393a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:20.564289 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41474efa-9ff0-470c-95f6-fa0ec357393a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:20.581777 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 548/1092] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Sep 14 11:47:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:20.587512 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb201f2-d14a-43a8-8483-0db58fe638ae None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:20.677740 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:20.693075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb201f2-d14a-43a8-8483-0db58fe638ae None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:20.727422 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993feee-ce09-4872-a38e-38373b46af49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:20.728022 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993feee-ce09-4872-a38e-38373b46af49 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:20.728306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993feee-ce09-4872-a38e-38373b46af49 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:20.728584 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993feee-ce09-4872-a38e-38373b46af49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:20.795868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993feee-ce09-4872-a38e-38373b46af49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:20.797285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993feee-ce09-4872-a38e-38373b46af49 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:20.884802 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:20.894298 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 545/1093] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:47:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:47:20.900137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41474efa-9ff0-470c-95f6-fa0ec357393a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:20.971518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:21.001690 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41474efa-9ff0-470c-95f6-fa0ec357393a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:21.035633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132cb958-0dce-43ee-a989-cf5d3fb8c036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:21.036260 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132cb958-0dce-43ee-a989-cf5d3fb8c036 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:21.036545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132cb958-0dce-43ee-a989-cf5d3fb8c036 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:21.036820 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132cb958-0dce-43ee-a989-cf5d3fb8c036 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:21.046746 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132cb958-0dce-43ee-a989-cf5d3fb8c036 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:21.048403 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132cb958-0dce-43ee-a989-cf5d3fb8c036 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:21.063135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-132cb958-0dce-43ee-a989-cf5d3fb8c036 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:47:21.063135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:21.063135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:21.063135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:21.063135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:21.063135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:21.063135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:21.063135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:21.063135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:21.063135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:21.063135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:21.063135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:21.063135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:47:21.063135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:47:21.063135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:47:21.063135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:47:21.064238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:21.064238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:21.064238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:21.064238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:21.064238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:21.064238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:21.064238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:21.064238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:21.064238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:21.064238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:21.064238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:21.064238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:21.064238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:21.064238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:21.064238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:21.064238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:21.064238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:47:21.065353 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:47:21.065353 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:47:21.065353 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:21.066023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 549/1094] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:21.069852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993feee-ce09-4872-a38e-38373b46af49 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:21.163981 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993feee-ce09-4872-a38e-38373b46af49 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:21.196453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7210b54-c6a1-4277-ae02-e436ed0fbfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:21.197050 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7210b54-c6a1-4277-ae02-e436ed0fbfbf None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:21.197329 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7210b54-c6a1-4277-ae02-e436ed0fbfbf None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:21.197600 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7210b54-c6a1-4277-ae02-e436ed0fbfbf None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:21.198290 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7210b54-c6a1-4277-ae02-e436ed0fbfbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:21.198592 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7210b54-c6a1-4277-ae02-e436ed0fbfbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:21.199966 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7210b54-c6a1-4277-ae02-e436ed0fbfbf None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:21.210682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 546/1095] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:21.216809 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132cb958-0dce-43ee-a989-cf5d3fb8c036 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:21.314206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132cb958-0dce-43ee-a989-cf5d3fb8c036 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:21.335977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:21.347319 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04828230-a6f2-49b0-8a4a-c00d208da7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:21.347939 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04828230-a6f2-49b0-8a4a-c00d208da7ad None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:21.354638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04828230-a6f2-49b0-8a4a-c00d208da7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:21.354638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04828230-a6f2-49b0-8a4a-c00d208da7ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:21.355646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04828230-a6f2-49b0-8a4a-c00d208da7ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:21.357085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04828230-a6f2-49b0-8a4a-c00d208da7ad None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:21.368333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-04828230-a6f2-49b0-8a4a-c00d208da7ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:47:21.368333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:21.368333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:21.368333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:21.368333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:21.368333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:21.368333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:21.368333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:21.368333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:21.368333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:21.368333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:21.368333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:21.368333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:47:21.368333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:47:21.368333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:47:21.369371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:47:21.369371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:21.369371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:21.369371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:21.369371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:21.369371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:21.369371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:21.369371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:21.369371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:21.369371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:21.369371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:21.369371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:21.369371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:21.369371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:21.369371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:21.369371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:21.369371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:21.370996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:47:21.370996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:47:21.370996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:47:21.370996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:47:21.370996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:47:21.370996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:47:21.370996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:47:21.370996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:21.370996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 550/1096] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:47:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:21.375782 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7210b54-c6a1-4277-ae02-e436ed0fbfbf None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:21.429336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:21.475760 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7210b54-c6a1-4277-ae02-e436ed0fbfbf None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:21.508312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab34d8e-7ae9-4ba2-b698-572f51337faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:21.508910 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab34d8e-7ae9-4ba2-b698-572f51337faf None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:21.509188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab34d8e-7ae9-4ba2-b698-572f51337faf None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:21.509458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab34d8e-7ae9-4ba2-b698-572f51337faf None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:21.510177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab34d8e-7ae9-4ba2-b698-572f51337faf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:21.510524 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab34d8e-7ae9-4ba2-b698-572f51337faf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:21.511929 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab34d8e-7ae9-4ba2-b698-572f51337faf None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:21.523480 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 547/1097] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:47:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:21.529785 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04828230-a6f2-49b0-8a4a-c00d208da7ad None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:21.629053 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04828230-a6f2-49b0-8a4a-c00d208da7ad None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:21.662957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b4632-3ec7-4229-aa9b-0c3bf7c5f90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:21.663565 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b4632-3ec7-4229-aa9b-0c3bf7c5f90a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:21.663851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b4632-3ec7-4229-aa9b-0c3bf7c5f90a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:21.664133 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b4632-3ec7-4229-aa9b-0c3bf7c5f90a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:21.664865 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b4632-3ec7-4229-aa9b-0c3bf7c5f90a None admin] RBAC: Adding query filter params (role.id=09d8286638fb49d8a4499662066fda1a, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=38b093bc4c1d4ebcbd2a6bb16b70c029) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:21.665180 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b4632-3ec7-4229-aa9b-0c3bf7c5f90a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:21.666805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b4632-3ec7-4229-aa9b-0c3bf7c5f90a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:21.672608 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 551/1098] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:47:21 2022] GET /identity/v3/role_assignments?user.id=38b093bc4c1d4ebcbd2a6bb16b70c029&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=09d8286638fb49d8a4499662066fda1a => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:22.047045 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932332e-f2c0-41d6-87a9-c378eca30eb3 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:22.047317 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932332e-f2c0-41d6-87a9-c378eca30eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:22.047520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932332e-f2c0-41d6-87a9-c378eca30eb3 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:22.049283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 548/1099] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:22.054630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0c402-8f2a-4027-ac99-19123bd3b74e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:22.055117 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0c402-8f2a-4027-ac99-19123bd3b74e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:22.055218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0c402-8f2a-4027-ac99-19123bd3b74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:22.074278 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-60d0c402-8f2a-4027-ac99-19123bd3b74e None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:22.075470 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-60d0c402-8f2a-4027-ac99-19123bd3b74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:22.118104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:22.143368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 552/1100] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:22.429525 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932332e-f2c0-41d6-87a9-c378eca30eb3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:22.535680 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932332e-f2c0-41d6-87a9-c378eca30eb3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:22.549285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:22.569105 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c5b16-3441-4c63-b8cf-547dbb6323b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:22.569725 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c5b16-3441-4c63-b8cf-547dbb6323b6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:22.570035 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c5b16-3441-4c63-b8cf-547dbb6323b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:22.570309 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c5b16-3441-4c63-b8cf-547dbb6323b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:22.579894 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c5b16-3441-4c63-b8cf-547dbb6323b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:47:22.584683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-aa1c5b16-3441-4c63-b8cf-547dbb6323b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:47:22.584683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:22.584683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:22.584683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:22.584683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:22.584683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:22.584683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:22.584683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:22.584683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:22.584683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:22.584683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:22.584683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:22.584683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:47:22.584683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:47:22.584683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:47:22.584683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:47:22.585928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:47:22.585928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:47:22.585928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:47:22.585928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:47:22.585928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:22.585928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:22.585928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:22.585928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:22.585928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:22.585928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:22.585928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:22.585928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:22.585928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:22.585928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:22.585928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:22.585928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:22.587117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:22.587117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:22.587117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:22.587117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:22.587117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:47:22.587117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:47:22.587117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:47:22.587117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:47:22.587117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:47:22.587117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:47:22.587117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:47:22.587117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:22.587117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 549/1101] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:47:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:22.592310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0c402-8f2a-4027-ac99-19123bd3b74e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:22.687951 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0c402-8f2a-4027-ac99-19123bd3b74e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:22.720481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161354f-c40c-48d6-87ef-559f575a95e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:22.721072 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161354f-c40c-48d6-87ef-559f575a95e9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:22.721355 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161354f-c40c-48d6-87ef-559f575a95e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:22.721634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161354f-c40c-48d6-87ef-559f575a95e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:22.722319 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161354f-c40c-48d6-87ef-559f575a95e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:22.722629 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161354f-c40c-48d6-87ef-559f575a95e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:22.723990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161354f-c40c-48d6-87ef-559f575a95e9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:22.732229 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 553/1102] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:47:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:22.738113 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c5b16-3441-4c63-b8cf-547dbb6323b6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:22.835245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c5b16-3441-4c63-b8cf-547dbb6323b6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:22.869253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3b3eb-204d-48b3-b5dd-77769a35c933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:22.869930 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3b3eb-204d-48b3-b5dd-77769a35c933 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:22.870218 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3b3eb-204d-48b3-b5dd-77769a35c933 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:22.870497 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3b3eb-204d-48b3-b5dd-77769a35c933 None admin] PATH_INFO: `/v3/users/623b5af81ef34c8cb7450151fb883146` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:22.871603 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3b3eb-204d-48b3-b5dd-77769a35c933 None admin] RBAC: Authorizing `identity:get_user(user_id=623b5af81ef34c8cb7450151fb883146)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:22.873025 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3b3eb-204d-48b3-b5dd-77769a35c933 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:22.874624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 550/1103] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:47:22 2022] GET /identity/v3/users/623b5af81ef34c8cb7450151fb883146 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:22.880594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161354f-c40c-48d6-87ef-559f575a95e9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:22.921196 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:22.981024 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161354f-c40c-48d6-87ef-559f575a95e9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:23.013820 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807c295-dc77-47a0-9ba0-5bec774d3427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:23.014445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807c295-dc77-47a0-9ba0-5bec774d3427 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:23.014734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807c295-dc77-47a0-9ba0-5bec774d3427 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:23.015011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807c295-dc77-47a0-9ba0-5bec774d3427 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:23.080238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807c295-dc77-47a0-9ba0-5bec774d3427 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:23.081676 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807c295-dc77-47a0-9ba0-5bec774d3427 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:23.118424 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:23.176745 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 554/1104] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:47:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:47:23.182755 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3b3eb-204d-48b3-b5dd-77769a35c933 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:23.224136 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:23.280343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3b3eb-204d-48b3-b5dd-77769a35c933 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:23.314112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf9530-6593-467c-891b-b3c883b9d22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:23.314727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf9530-6593-467c-891b-b3c883b9d22d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:23.315011 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf9530-6593-467c-891b-b3c883b9d22d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:23.315288 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf9530-6593-467c-891b-b3c883b9d22d None admin] PATH_INFO: `/v3/projects/3c8ee4546b6c44c78943e2cb9864ec15` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:23.316397 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf9530-6593-467c-891b-b3c883b9d22d None admin] RBAC: Authorizing `identity:get_project(project_id=3c8ee4546b6c44c78943e2cb9864ec15)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:23.317810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf9530-6593-467c-891b-b3c883b9d22d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:23.319438 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 551/1105] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:47:23 2022] GET /identity/v3/projects/3c8ee4546b6c44c78943e2cb9864ec15 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:23.326050 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807c295-dc77-47a0-9ba0-5bec774d3427 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:23.422921 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807c295-dc77-47a0-9ba0-5bec774d3427 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:23.457336 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a5401-471e-4d76-b1f7-85fd9f02332f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:23.457954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a5401-471e-4d76-b1f7-85fd9f02332f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:23.458241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a5401-471e-4d76-b1f7-85fd9f02332f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:23.458520 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a5401-471e-4d76-b1f7-85fd9f02332f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:23.459241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a5401-471e-4d76-b1f7-85fd9f02332f None admin] RBAC: Adding query filter params (role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.project.id=3c8ee4546b6c44c78943e2cb9864ec15, user.id=623b5af81ef34c8cb7450151fb883146) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:23.459554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a5401-471e-4d76-b1f7-85fd9f02332f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:23.460979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a5401-471e-4d76-b1f7-85fd9f02332f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:23.466575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 555/1106] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:47:23 2022] GET /identity/v3/role_assignments?user.id=623b5af81ef34c8cb7450151fb883146&scope.project.id=3c8ee4546b6c44c78943e2cb9864ec15&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:25.627319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f222c41-0065-4c8f-bcab-b67020b3519d None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:25.627879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f222c41-0065-4c8f-bcab-b67020b3519d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:25.627879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f222c41-0065-4c8f-bcab-b67020b3519d None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:25.629235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 552/1107] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:25.634427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583b135-153f-4d27-9e3d-5ac24aae847a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:25.634810 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583b135-153f-4d27-9e3d-5ac24aae847a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:25.634984 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583b135-153f-4d27-9e3d-5ac24aae847a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:25.641164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:25.653690 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9583b135-153f-4d27-9e3d-5ac24aae847a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:25.656246 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9583b135-153f-4d27-9e3d-5ac24aae847a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:25.719376 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 556/1108] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:25.987544 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f222c41-0065-4c8f-bcab-b67020b3519d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:26.087529 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f222c41-0065-4c8f-bcab-b67020b3519d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:26.098390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:26.122273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e5d50-7b32-4c76-94e8-6f6261f881ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:26.122892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e5d50-7b32-4c76-94e8-6f6261f881ac None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:26.123176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e5d50-7b32-4c76-94e8-6f6261f881ac None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:26.123453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e5d50-7b32-4c76-94e8-6f6261f881ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:26.124559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e5d50-7b32-4c76-94e8-6f6261f881ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:26.126086 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e5d50-7b32-4c76-94e8-6f6261f881ac None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:26.127650 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 553/1109] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:26.133579 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583b135-153f-4d27-9e3d-5ac24aae847a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:26.229996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583b135-153f-4d27-9e3d-5ac24aae847a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:26.258597 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:26.263807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5935232-4034-4cd1-b963-fa86e60c7f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:26.264405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5935232-4034-4cd1-b963-fa86e60c7f8b None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:26.264687 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5935232-4034-4cd1-b963-fa86e60c7f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:26.264965 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5935232-4034-4cd1-b963-fa86e60c7f8b None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:26.265611 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5935232-4034-4cd1-b963-fa86e60c7f8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:26.267074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5935232-4034-4cd1-b963-fa86e60c7f8b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:26.407286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 557/1110] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 14 11:47:26 2022] POST /identity/v3/users => generated 286 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 11:47:28.600204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edffa7-976f-4ccc-ad3c-16e197eedf87 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:28.600762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edffa7-976f-4ccc-ad3c-16e197eedf87 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:28.600762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edffa7-976f-4ccc-ad3c-16e197eedf87 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:28.602155 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 554/1111] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:28.608085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a2f69-92fb-4b74-a1d0-98e58e8c0bf1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:28.608511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a2f69-92fb-4b74-a1d0-98e58e8c0bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:28.608649 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a2f69-92fb-4b74-a1d0-98e58e8c0bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:28.627227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-836a2f69-92fb-4b74-a1d0-98e58e8c0bf1 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:28.628534 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-836a2f69-92fb-4b74-a1d0-98e58e8c0bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:28.696650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 558/1112] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:28.977904 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33edffa7-976f-4ccc-ad3c-16e197eedf87 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:29.077851 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33edffa7-976f-4ccc-ad3c-16e197eedf87 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:29.112489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11e15c-e85f-43bd-82f3-f31071b0ae2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:29.113119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11e15c-e85f-43bd-82f3-f31071b0ae2b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:29.113401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11e15c-e85f-43bd-82f3-f31071b0ae2b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:29.113678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11e15c-e85f-43bd-82f3-f31071b0ae2b None admin] PATH_INFO: `/v3/roles/1ab8c505dc344769abc347f66e529527` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:29.115088 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11e15c-e85f-43bd-82f3-f31071b0ae2b None admin] RBAC: Authorizing `identity:get_role(role_id=1ab8c505dc344769abc347f66e529527)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:29.116454 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11e15c-e85f-43bd-82f3-f31071b0ae2b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:29.117807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 555/1113] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:47:28 2022] GET /identity/v3/roles/1ab8c505dc344769abc347f66e529527 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:29.123695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a2f69-92fb-4b74-a1d0-98e58e8c0bf1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:29.221779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a2f69-92fb-4b74-a1d0-98e58e8c0bf1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:29.238902 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:29.255752 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415fb4c0-5fd4-44ed-a548-f05bf1e013e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:29.256382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fb4c0-5fd4-44ed-a548-f05bf1e013e2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:29.256668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fb4c0-5fd4-44ed-a548-f05bf1e013e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:29.256948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fb4c0-5fd4-44ed-a548-f05bf1e013e2 None admin] PATH_INFO: `/v3/users/0e698a2159654ccab20bf248b0afe857` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:29.271284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415fb4c0-5fd4-44ed-a548-f05bf1e013e2 None admin] RBAC: Authorizing `identity:get_user(user_id=0e698a2159654ccab20bf248b0afe857)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:29.272728 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415fb4c0-5fd4-44ed-a548-f05bf1e013e2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:29.274372 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 559/1114] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:47:29 2022] GET /identity/v3/users/0e698a2159654ccab20bf248b0afe857 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:29.280463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11e15c-e85f-43bd-82f3-f31071b0ae2b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:29.327556 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:29.379660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11e15c-e85f-43bd-82f3-f31071b0ae2b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:29.412718 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0715de-d6ec-4098-8eab-6d464e9ee5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:29.413314 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0715de-d6ec-4098-8eab-6d464e9ee5b3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:29.413597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0715de-d6ec-4098-8eab-6d464e9ee5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:29.413906 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0715de-d6ec-4098-8eab-6d464e9ee5b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:29.478480 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0715de-d6ec-4098-8eab-6d464e9ee5b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:29.480282 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0715de-d6ec-4098-8eab-6d464e9ee5b3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:29.522605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:29.576147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 556/1115] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:47:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:47:29.582111 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415fb4c0-5fd4-44ed-a548-f05bf1e013e2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:29.679754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415fb4c0-5fd4-44ed-a548-f05bf1e013e2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:29.726047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925163e-135d-4e9e-ae3e-bd5e03703c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:29.726047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925163e-135d-4e9e-ae3e-bd5e03703c3e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:29.726047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925163e-135d-4e9e-ae3e-bd5e03703c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:29.726047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925163e-135d-4e9e-ae3e-bd5e03703c3e None admin] PATH_INFO: `/v3/projects/3c8ee4546b6c44c78943e2cb9864ec15` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:29.726941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925163e-135d-4e9e-ae3e-bd5e03703c3e None admin] RBAC: Authorizing `identity:get_project(project_id=3c8ee4546b6c44c78943e2cb9864ec15)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:29.726941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925163e-135d-4e9e-ae3e-bd5e03703c3e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:29.726941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 560/1116] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:47:29 2022] GET /identity/v3/projects/3c8ee4546b6c44c78943e2cb9864ec15 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:29.729669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0715de-d6ec-4098-8eab-6d464e9ee5b3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:29.839689 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0715de-d6ec-4098-8eab-6d464e9ee5b3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:29.873905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132785b-eef0-4d84-b7e7-564c687d26c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:29.874505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132785b-eef0-4d84-b7e7-564c687d26c7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:29.874790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132785b-eef0-4d84-b7e7-564c687d26c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:29.875085 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132785b-eef0-4d84-b7e7-564c687d26c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:29.875804 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132785b-eef0-4d84-b7e7-564c687d26c7 None admin] RBAC: Adding query filter params (role.id=1ab8c505dc344769abc347f66e529527, scope.project.id=3c8ee4546b6c44c78943e2cb9864ec15, user.id=0e698a2159654ccab20bf248b0afe857) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:29.876140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132785b-eef0-4d84-b7e7-564c687d26c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:29.877566 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132785b-eef0-4d84-b7e7-564c687d26c7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:29.884228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 557/1117] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:47:29 2022] GET /identity/v3/role_assignments?user.id=0e698a2159654ccab20bf248b0afe857&scope.project.id=3c8ee4546b6c44c78943e2cb9864ec15&role.id=1ab8c505dc344769abc347f66e529527 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:32.016843 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79462da-1d09-460f-88b1-5eb42e9f7db6 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:32.017402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79462da-1d09-460f-88b1-5eb42e9f7db6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:32.017402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79462da-1d09-460f-88b1-5eb42e9f7db6 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:32.018824 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 561/1118] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:32.024008 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bdfc8-141e-4921-b421-f9f22ae0570f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:32.024381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bdfc8-141e-4921-b421-f9f22ae0570f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:32.024571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bdfc8-141e-4921-b421-f9f22ae0570f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:32.043803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0e3bdfc8-141e-4921-b421-f9f22ae0570f None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:32.044987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0e3bdfc8-141e-4921-b421-f9f22ae0570f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:32.046934 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:32.110669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 558/1119] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:32.396253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79462da-1d09-460f-88b1-5eb42e9f7db6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:32.442527 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:32.497008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79462da-1d09-460f-88b1-5eb42e9f7db6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:32.531809 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501d693-43a5-4cea-8c58-43fabeece6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:32.532429 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501d693-43a5-4cea-8c58-43fabeece6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:32.532713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501d693-43a5-4cea-8c58-43fabeece6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:32.532992 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501d693-43a5-4cea-8c58-43fabeece6e0 None admin] PATH_INFO: `/v3/roles/1ab8c505dc344769abc347f66e529527` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:32.534480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501d693-43a5-4cea-8c58-43fabeece6e0 None admin] RBAC: Authorizing `identity:get_role(role_id=1ab8c505dc344769abc347f66e529527)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:32.535847 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501d693-43a5-4cea-8c58-43fabeece6e0 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:32.537161 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 562/1120] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:47:32 2022] GET /identity/v3/roles/1ab8c505dc344769abc347f66e529527 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:32.543144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bdfc8-141e-4921-b421-f9f22ae0570f None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:32.639329 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bdfc8-141e-4921-b421-f9f22ae0570f None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:32.672091 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f576241-65a4-471e-8841-c2bd064ab7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:32.672645 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f576241-65a4-471e-8841-c2bd064ab7ca None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:32.673198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f576241-65a4-471e-8841-c2bd064ab7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:32.673473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f576241-65a4-471e-8841-c2bd064ab7ca None admin] PATH_INFO: `/v3/users/0e698a2159654ccab20bf248b0afe857` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:32.674641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f576241-65a4-471e-8841-c2bd064ab7ca None admin] RBAC: Authorizing `identity:get_user(user_id=0e698a2159654ccab20bf248b0afe857)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:32.676077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f576241-65a4-471e-8841-c2bd064ab7ca None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:32.677631 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 559/1121] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:47:32 2022] GET /identity/v3/users/0e698a2159654ccab20bf248b0afe857 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:32.683445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501d693-43a5-4cea-8c58-43fabeece6e0 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:32.781304 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501d693-43a5-4cea-8c58-43fabeece6e0 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:32.794057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:32.814426 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664eccb-1c67-4796-bfc0-c1f7e5c6bc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:32.815034 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664eccb-1c67-4796-bfc0-c1f7e5c6bc8b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:32.815314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664eccb-1c67-4796-bfc0-c1f7e5c6bc8b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:32.815587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664eccb-1c67-4796-bfc0-c1f7e5c6bc8b None admin] PATH_INFO: `/v3/projects/3c8ee4546b6c44c78943e2cb9864ec15` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:32.816683 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5664eccb-1c67-4796-bfc0-c1f7e5c6bc8b None admin] RBAC: Authorizing `identity:get_project(project_id=3c8ee4546b6c44c78943e2cb9864ec15)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:32.818247 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5664eccb-1c67-4796-bfc0-c1f7e5c6bc8b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:32.820009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 563/1122] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:47:32 2022] GET /identity/v3/projects/3c8ee4546b6c44c78943e2cb9864ec15 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:32.826480 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f576241-65a4-471e-8841-c2bd064ab7ca None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:32.835303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:32.923633 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f576241-65a4-471e-8841-c2bd064ab7ca None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:32.956119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bdfec0-fc12-43ec-873a-ff206a9cee96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:32.956764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdfec0-fc12-43ec-873a-ff206a9cee96 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:32.957048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdfec0-fc12-43ec-873a-ff206a9cee96 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:32.957324 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdfec0-fc12-43ec-873a-ff206a9cee96 None admin] PATH_INFO: `/v3/projects/3c8ee4546b6c44c78943e2cb9864ec15/users/0e698a2159654ccab20bf248b0afe857/roles/1ab8c505dc344769abc347f66e529527` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:32.959521 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bdfec0-fc12-43ec-873a-ff206a9cee96 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c8ee4546b6c44c78943e2cb9864ec15, user_id=0e698a2159654ccab20bf248b0afe857, role_id=1ab8c505dc344769abc347f66e529527)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:32.960939 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bdfec0-fc12-43ec-873a-ff206a9cee96 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:33.029445 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 560/1123] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:47:32 2022] PUT /identity/v3/projects/3c8ee4546b6c44c78943e2cb9864ec15/users/0e698a2159654ccab20bf248b0afe857/roles/1ab8c505dc344769abc347f66e529527 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:47:35.205346 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b9493-b0fe-47f2-9499-31cccb014cb6 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:35.205938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b9493-b0fe-47f2-9499-31cccb014cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:35.205938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b9493-b0fe-47f2-9499-31cccb014cb6 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:35.207276 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 564/1124] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:35.212555 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d967b7-ace3-40b0-9af8-b46fb416b28e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:35.213456 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d967b7-ace3-40b0-9af8-b46fb416b28e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:35.213593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d967b7-ace3-40b0-9af8-b46fb416b28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:35.231766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e8d967b7-ace3-40b0-9af8-b46fb416b28e None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:35.232934 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e8d967b7-ace3-40b0-9af8-b46fb416b28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:35.288539 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:35.296069 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 561/1125] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:35.561832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b9493-b0fe-47f2-9499-31cccb014cb6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:35.660516 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b9493-b0fe-47f2-9499-31cccb014cb6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:35.694365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4becd369-1316-4dc9-8b3b-16f9c0e7cd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:35.694978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becd369-1316-4dc9-8b3b-16f9c0e7cd08 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:35.695260 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becd369-1316-4dc9-8b3b-16f9c0e7cd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:35.695535 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becd369-1316-4dc9-8b3b-16f9c0e7cd08 None admin] PATH_INFO: `/v3/roles/1ab8c505dc344769abc347f66e529527` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:35.696917 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4becd369-1316-4dc9-8b3b-16f9c0e7cd08 None admin] RBAC: Authorizing `identity:get_role(role_id=1ab8c505dc344769abc347f66e529527)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:35.698338 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4becd369-1316-4dc9-8b3b-16f9c0e7cd08 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:35.699638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 565/1126] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:47:35 2022] GET /identity/v3/roles/1ab8c505dc344769abc347f66e529527 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:35.705371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d967b7-ace3-40b0-9af8-b46fb416b28e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:35.800536 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d967b7-ace3-40b0-9af8-b46fb416b28e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:35.833285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29cf11-b078-416f-b1c5-7e81e2811b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:35.833938 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29cf11-b078-416f-b1c5-7e81e2811b4a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:35.834221 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29cf11-b078-416f-b1c5-7e81e2811b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:35.834493 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29cf11-b078-416f-b1c5-7e81e2811b4a None admin] PATH_INFO: `/v3/users/0e698a2159654ccab20bf248b0afe857` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:35.835600 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29cf11-b078-416f-b1c5-7e81e2811b4a None admin] RBAC: Authorizing `identity:get_user(user_id=0e698a2159654ccab20bf248b0afe857)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:35.837020 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29cf11-b078-416f-b1c5-7e81e2811b4a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:35.838641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 562/1127] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:47:35 2022] GET /identity/v3/users/0e698a2159654ccab20bf248b0afe857 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:35.844489 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4becd369-1316-4dc9-8b3b-16f9c0e7cd08 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:35.892916 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:35.944925 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4becd369-1316-4dc9-8b3b-16f9c0e7cd08 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:35.977866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cee96-c692-4557-bb72-2ee90c27e520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:35.978569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cee96-c692-4557-bb72-2ee90c27e520 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:35.979340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cee96-c692-4557-bb72-2ee90c27e520 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:35.979612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cee96-c692-4557-bb72-2ee90c27e520 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:36.043986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511cee96-c692-4557-bb72-2ee90c27e520 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:36.045406 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511cee96-c692-4557-bb72-2ee90c27e520 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:36.090005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:36.138852 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 566/1128] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:47:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:47:36.144835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29cf11-b078-416f-b1c5-7e81e2811b4a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:36.204737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:36.240628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29cf11-b078-416f-b1c5-7e81e2811b4a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:36.272620 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc67402-e4d1-450f-923f-1bccf44d13fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:36.273231 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc67402-e4d1-450f-923f-1bccf44d13fe None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:36.273513 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc67402-e4d1-450f-923f-1bccf44d13fe None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:36.273789 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc67402-e4d1-450f-923f-1bccf44d13fe None admin] PATH_INFO: `/v3/projects/3c8ee4546b6c44c78943e2cb9864ec15` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:36.274920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc67402-e4d1-450f-923f-1bccf44d13fe None admin] RBAC: Authorizing `identity:get_project(project_id=3c8ee4546b6c44c78943e2cb9864ec15)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:36.276859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc67402-e4d1-450f-923f-1bccf44d13fe None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:36.278473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 563/1129] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:47:36 2022] GET /identity/v3/projects/3c8ee4546b6c44c78943e2cb9864ec15 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:36.284405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cee96-c692-4557-bb72-2ee90c27e520 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:36.379648 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cee96-c692-4557-bb72-2ee90c27e520 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:36.413500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff38a94-9cca-405d-81a8-084ad3425c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:36.414119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff38a94-9cca-405d-81a8-084ad3425c04 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:36.414401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff38a94-9cca-405d-81a8-084ad3425c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:36.414675 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff38a94-9cca-405d-81a8-084ad3425c04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:36.415393 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff38a94-9cca-405d-81a8-084ad3425c04 None admin] RBAC: Adding query filter params (role.id=1ab8c505dc344769abc347f66e529527, scope.project.id=3c8ee4546b6c44c78943e2cb9864ec15, user.id=0e698a2159654ccab20bf248b0afe857) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:36.415699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff38a94-9cca-405d-81a8-084ad3425c04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:36.417154 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff38a94-9cca-405d-81a8-084ad3425c04 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:36.422976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 567/1130] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:47:36 2022] GET /identity/v3/role_assignments?user.id=0e698a2159654ccab20bf248b0afe857&scope.project.id=3c8ee4546b6c44c78943e2cb9864ec15&role.id=1ab8c505dc344769abc347f66e529527 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:38.564887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41576f8-3531-40e8-8bb7-eeae176a1292 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:38.565414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41576f8-3531-40e8-8bb7-eeae176a1292 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:38.565414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41576f8-3531-40e8-8bb7-eeae176a1292 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:38.566798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 564/1131] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:38.571932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648f88a-fa62-4e15-8d2d-536704ec16b6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:38.572868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648f88a-fa62-4e15-8d2d-536704ec16b6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:38.573004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648f88a-fa62-4e15-8d2d-536704ec16b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:38.591566 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1648f88a-fa62-4e15-8d2d-536704ec16b6 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:38.592760 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1648f88a-fa62-4e15-8d2d-536704ec16b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:38.599993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:38.656981 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 568/1132] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:38.921267 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41576f8-3531-40e8-8bb7-eeae176a1292 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:39.020996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41576f8-3531-40e8-8bb7-eeae176a1292 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:39.049018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:39.053672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7a9f1-28da-43f4-9d43-d62f52673220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:39.054318 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7a9f1-28da-43f4-9d43-d62f52673220 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:39.054604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7a9f1-28da-43f4-9d43-d62f52673220 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:39.054880 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7a9f1-28da-43f4-9d43-d62f52673220 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:39.055986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7a9f1-28da-43f4-9d43-d62f52673220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:39.057962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7a9f1-28da-43f4-9d43-d62f52673220 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:39.059504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 565/1133] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:39.065294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648f88a-fa62-4e15-8d2d-536704ec16b6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:39.160866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648f88a-fa62-4e15-8d2d-536704ec16b6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:39.201044 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910ea0d-4139-4b9d-bdc6-a69c7e783d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:39.201635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910ea0d-4139-4b9d-bdc6-a69c7e783d53 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:39.201944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910ea0d-4139-4b9d-bdc6-a69c7e783d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:39.202223 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910ea0d-4139-4b9d-bdc6-a69c7e783d53 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:39.202882 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910ea0d-4139-4b9d-bdc6-a69c7e783d53 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:39.204293 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910ea0d-4139-4b9d-bdc6-a69c7e783d53 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:39.218080 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:39.299440 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 569/1134] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 14 11:47:39 2022] POST /identity/v3/projects => generated 311 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:47:41.448159 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90007200-ee77-4c06-b429-549cb6b901c6 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:41.448687 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90007200-ee77-4c06-b429-549cb6b901c6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:41.448793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90007200-ee77-4c06-b429-549cb6b901c6 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:41.450091 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 566/1135] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:41.455344 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fafa7-0a55-47d9-a012-04e393880a21 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:41.455789 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fafa7-0a55-47d9-a012-04e393880a21 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:41.455911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fafa7-0a55-47d9-a012-04e393880a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:41.474154 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-515fafa7-0a55-47d9-a012-04e393880a21 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:41.475335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-515fafa7-0a55-47d9-a012-04e393880a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:41.538358 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 570/1136] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:41.810605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90007200-ee77-4c06-b429-549cb6b901c6 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:41.908798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90007200-ee77-4c06-b429-549cb6b901c6 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:41.942034 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a82e74-7f23-4d55-bc3a-7ce5403c58dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:41.956152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a82e74-7f23-4d55-bc3a-7ce5403c58dd None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:41.956152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a82e74-7f23-4d55-bc3a-7ce5403c58dd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:41.956152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a82e74-7f23-4d55-bc3a-7ce5403c58dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:41.956152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a82e74-7f23-4d55-bc3a-7ce5403c58dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:41.956152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a82e74-7f23-4d55-bc3a-7ce5403c58dd None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:41.956152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 567/1137] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:47:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:41.956668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fafa7-0a55-47d9-a012-04e393880a21 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:42.050774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fafa7-0a55-47d9-a012-04e393880a21 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:42.074490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:42.084691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd83e5-0204-42cc-a094-13012df8234c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:42.085295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd83e5-0204-42cc-a094-13012df8234c None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:42.085591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd83e5-0204-42cc-a094-13012df8234c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:42.085891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd83e5-0204-42cc-a094-13012df8234c None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:42.086564 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd83e5-0204-42cc-a094-13012df8234c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:42.087993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd83e5-0204-42cc-a094-13012df8234c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:42.161206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 571/1138] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 14 11:47:41 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:47:44.323804 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345382e4-c50f-45ba-bcda-03a2a82da0a4 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:44.324341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345382e4-c50f-45ba-bcda-03a2a82da0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:44.324341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345382e4-c50f-45ba-bcda-03a2a82da0a4 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:44.325564 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 568/1139] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:44.330918 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d67d7-78f6-439b-88de-acbee1c0f4b5 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:44.331324 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d67d7-78f6-439b-88de-acbee1c0f4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:44.331461 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d67d7-78f6-439b-88de-acbee1c0f4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:44.349155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-479d67d7-78f6-439b-88de-acbee1c0f4b5 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:44.350368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-479d67d7-78f6-439b-88de-acbee1c0f4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:44.414237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 572/1140] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:44.680854 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345382e4-c50f-45ba-bcda-03a2a82da0a4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:44.747116 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:44.780074 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345382e4-c50f-45ba-bcda-03a2a82da0a4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:44.813586 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf6ecc-1870-42f5-b0a6-8cd16ea3025c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:44.814240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf6ecc-1870-42f5-b0a6-8cd16ea3025c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:44.814523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf6ecc-1870-42f5-b0a6-8cd16ea3025c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:44.814797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf6ecc-1870-42f5-b0a6-8cd16ea3025c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:44.825250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf6ecc-1870-42f5-b0a6-8cd16ea3025c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:47:44.830065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-90bf6ecc-1870-42f5-b0a6-8cd16ea3025c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:47:44.830065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:44.830065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:44.830065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:44.830065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:44.830065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:44.830065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:44.830065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:44.830065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:44.830065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:44.830065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:44.830065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:44.830065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:47:44.830065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:47:44.830065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:47:44.830065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:47:44.831178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:47:44.831178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:47:44.831178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:47:44.831178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:47:44.831178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:44.831178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:44.831178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:44.831178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:44.831178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:44.831178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:44.831178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:44.831178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:44.831178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:44.831178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:44.831178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:44.831178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:44.832104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:44.832104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:44.832104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:44.832104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:44.832104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:47:44.832104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:47:44.832104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:47:44.832104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:47:44.832104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:47:44.832104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:47:44.832104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:47:44.832104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:44.832104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 569/1141] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:47:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:44.837163 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d67d7-78f6-439b-88de-acbee1c0f4b5 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:44.936302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d67d7-78f6-439b-88de-acbee1c0f4b5 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:44.947586 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:44.970707 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c773de6-8463-4299-b8f1-45741b103160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:44.971436 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c773de6-8463-4299-b8f1-45741b103160 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:44.971660 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c773de6-8463-4299-b8f1-45741b103160 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:44.971939 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c773de6-8463-4299-b8f1-45741b103160 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:44.972605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c773de6-8463-4299-b8f1-45741b103160 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:44.978068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c773de6-8463-4299-b8f1-45741b103160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:44.978068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c773de6-8463-4299-b8f1-45741b103160 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:44.982255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 573/1142] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:47:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:44.988139 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf6ecc-1870-42f5-b0a6-8cd16ea3025c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:45.178354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf6ecc-1870-42f5-b0a6-8cd16ea3025c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:45.204252 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:45.211617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7dd79-4011-4f4d-97bf-144ecad82c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:45.218448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7dd79-4011-4f4d-97bf-144ecad82c83 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:45.218448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7dd79-4011-4f4d-97bf-144ecad82c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:45.218448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7dd79-4011-4f4d-97bf-144ecad82c83 None admin] PATH_INFO: `/v3/users/f5e8ee18498647e5b3ac9afa31d37453` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:45.225140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7dd79-4011-4f4d-97bf-144ecad82c83 None admin] RBAC: Authorizing `identity:get_user(user_id=f5e8ee18498647e5b3ac9afa31d37453)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:45.226596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7dd79-4011-4f4d-97bf-144ecad82c83 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:45.228171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 570/1143] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:47:44 2022] GET /identity/v3/users/f5e8ee18498647e5b3ac9afa31d37453 => generated 286 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:45.234026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c773de6-8463-4299-b8f1-45741b103160 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:45.332131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c773de6-8463-4299-b8f1-45741b103160 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:45.364758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6247a-7e1b-435d-8c62-9c2166f56078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:45.365353 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6247a-7e1b-435d-8c62-9c2166f56078 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:45.365637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6247a-7e1b-435d-8c62-9c2166f56078 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:45.365944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6247a-7e1b-435d-8c62-9c2166f56078 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:45.405165 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:45.431348 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6247a-7e1b-435d-8c62-9c2166f56078 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:45.433399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6247a-7e1b-435d-8c62-9c2166f56078 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:45.528625 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 574/1144] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:47:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:47:45.534502 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7dd79-4011-4f4d-97bf-144ecad82c83 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:45.630763 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7dd79-4011-4f4d-97bf-144ecad82c83 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:45.664640 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0cd213-74f4-4e70-94ee-14badd655423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:45.665254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cd213-74f4-4e70-94ee-14badd655423 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:45.665534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cd213-74f4-4e70-94ee-14badd655423 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:45.665805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cd213-74f4-4e70-94ee-14badd655423 None admin] PATH_INFO: `/v3/projects/2b25126932324dd6ad035447cd1553d6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:45.667035 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0cd213-74f4-4e70-94ee-14badd655423 None admin] RBAC: Authorizing `identity:get_project(project_id=2b25126932324dd6ad035447cd1553d6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:45.668448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0cd213-74f4-4e70-94ee-14badd655423 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:45.670077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 571/1145] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:47:45 2022] GET /identity/v3/projects/2b25126932324dd6ad035447cd1553d6 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:45.676349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6247a-7e1b-435d-8c62-9c2166f56078 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:45.743875 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:45.773625 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6247a-7e1b-435d-8c62-9c2166f56078 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:45.807528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422809a-dd5d-4a43-a18b-106d7351713e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:45.808049 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422809a-dd5d-4a43-a18b-106d7351713e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:45.808331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422809a-dd5d-4a43-a18b-106d7351713e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:45.808605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422809a-dd5d-4a43-a18b-106d7351713e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:45.809352 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422809a-dd5d-4a43-a18b-106d7351713e None admin] RBAC: Adding query filter params (role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.project.id=2b25126932324dd6ad035447cd1553d6, user.id=f5e8ee18498647e5b3ac9afa31d37453) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:45.809659 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422809a-dd5d-4a43-a18b-106d7351713e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:45.811110 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422809a-dd5d-4a43-a18b-106d7351713e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:45.816345 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 575/1146] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:47:45 2022] GET /identity/v3/role_assignments?user.id=f5e8ee18498647e5b3ac9afa31d37453&scope.project.id=2b25126932324dd6ad035447cd1553d6&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:47.905667 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3657a66-3cbf-4829-9436-042205a6c698 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:47.906240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3657a66-3cbf-4829-9436-042205a6c698 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:47.906240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3657a66-3cbf-4829-9436-042205a6c698 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:47.907609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 572/1147] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:47.912798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef7b84-b998-4892-893c-88cd0aba7a70 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:47.913196 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef7b84-b998-4892-893c-88cd0aba7a70 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:47.913364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef7b84-b998-4892-893c-88cd0aba7a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:47.932740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-0aef7b84-b998-4892-893c-88cd0aba7a70 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:47.933976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0aef7b84-b998-4892-893c-88cd0aba7a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:48.000756 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 576/1148] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:48.267683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3657a66-3cbf-4829-9436-042205a6c698 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:48.323965 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:48.367114 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3657a66-3cbf-4829-9436-042205a6c698 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:48.399548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98466cb9-704a-47b9-acfe-56cfd15b2b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:48.400193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98466cb9-704a-47b9-acfe-56cfd15b2b25 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:48.400474 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98466cb9-704a-47b9-acfe-56cfd15b2b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:48.401316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98466cb9-704a-47b9-acfe-56cfd15b2b25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:48.410972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98466cb9-704a-47b9-acfe-56cfd15b2b25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:47:48.415714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-98466cb9-704a-47b9-acfe-56cfd15b2b25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:47:48.415714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:48.415714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:48.415714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:48.415714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:48.415714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:48.415714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:48.415714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:48.415714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:48.415714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:48.415714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:48.415714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:48.415714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:47:48.415714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:47:48.415714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:47:48.415714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:47:48.416822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:47:48.416822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:47:48.416822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:47:48.416822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:47:48.416822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:48.416822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:48.416822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:48.416822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:48.416822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:48.416822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:48.416822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:48.416822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:48.416822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:48.416822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:48.416822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:48.416822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:48.417728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:48.417728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:48.417728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:48.417728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:48.417728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:47:48.417728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:47:48.417728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:47:48.417728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:47:48.417728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:47:48.417728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:47:48.417728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:47:48.417728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:48.417728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 573/1149] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:47:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:48.423090 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef7b84-b998-4892-893c-88cd0aba7a70 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:48.486035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:48.519953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef7b84-b998-4892-893c-88cd0aba7a70 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:48.553200 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc64ce-479e-4a9e-9f98-59a6a0e6f2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:48.553799 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc64ce-479e-4a9e-9f98-59a6a0e6f2af None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:48.554115 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc64ce-479e-4a9e-9f98-59a6a0e6f2af None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:48.554399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc64ce-479e-4a9e-9f98-59a6a0e6f2af None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:48.555062 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc64ce-479e-4a9e-9f98-59a6a0e6f2af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:48.555371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc64ce-479e-4a9e-9f98-59a6a0e6f2af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:48.556824 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc64ce-479e-4a9e-9f98-59a6a0e6f2af None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:48.564613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 577/1150] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:47:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:48.570481 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98466cb9-704a-47b9-acfe-56cfd15b2b25 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:48.666364 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98466cb9-704a-47b9-acfe-56cfd15b2b25 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:48.679858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:48.700583 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39a066-ade2-4da6-b3a5-b3a4137d1cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:48.701196 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39a066-ade2-4da6-b3a5-b3a4137d1cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:48.701478 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39a066-ade2-4da6-b3a5-b3a4137d1cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:48.701750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39a066-ade2-4da6-b3a5-b3a4137d1cd9 None admin] PATH_INFO: `/v3/users/f5e8ee18498647e5b3ac9afa31d37453` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:48.702895 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39a066-ade2-4da6-b3a5-b3a4137d1cd9 None admin] RBAC: Authorizing `identity:get_user(user_id=f5e8ee18498647e5b3ac9afa31d37453)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:48.704300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39a066-ade2-4da6-b3a5-b3a4137d1cd9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:48.705863 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 574/1151] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:47:48 2022] GET /identity/v3/users/f5e8ee18498647e5b3ac9afa31d37453 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:48.711692 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc64ce-479e-4a9e-9f98-59a6a0e6f2af None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:48.809608 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc64ce-479e-4a9e-9f98-59a6a0e6f2af None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:48.835528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:48.843466 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60745830-fccd-4224-aa03-fb9a333f93fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:48.849198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60745830-fccd-4224-aa03-fb9a333f93fa None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:48.849198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60745830-fccd-4224-aa03-fb9a333f93fa None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:48.849198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60745830-fccd-4224-aa03-fb9a333f93fa None admin] PATH_INFO: `/v3/projects/2b25126932324dd6ad035447cd1553d6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:48.849198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60745830-fccd-4224-aa03-fb9a333f93fa None admin] RBAC: Authorizing `identity:get_project(project_id=2b25126932324dd6ad035447cd1553d6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:48.849198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60745830-fccd-4224-aa03-fb9a333f93fa None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:48.849198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 578/1152] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:47:48 2022] GET /identity/v3/projects/2b25126932324dd6ad035447cd1553d6 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:48.854995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39a066-ade2-4da6-b3a5-b3a4137d1cd9 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:48.951547 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39a066-ade2-4da6-b3a5-b3a4137d1cd9 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:48.983685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b647b-ffbf-46b7-86de-8cd07d968a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:48.984333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b647b-ffbf-46b7-86de-8cd07d968a05 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:48.984617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b647b-ffbf-46b7-86de-8cd07d968a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:48.984893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b647b-ffbf-46b7-86de-8cd07d968a05 None admin] PATH_INFO: `/v3/projects/2b25126932324dd6ad035447cd1553d6/users/f5e8ee18498647e5b3ac9afa31d37453/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:48.986794 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b647b-ffbf-46b7-86de-8cd07d968a05 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b25126932324dd6ad035447cd1553d6, user_id=f5e8ee18498647e5b3ac9afa31d37453, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:48.988209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b647b-ffbf-46b7-86de-8cd07d968a05 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:49.058827 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 575/1153] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:47:48 2022] PUT /identity/v3/projects/2b25126932324dd6ad035447cd1553d6/users/f5e8ee18498647e5b3ac9afa31d37453/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:47:51.139722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4dff9-7795-48c5-b372-2f9b61f60156 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:51.140253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4dff9-7795-48c5-b372-2f9b61f60156 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:51.140253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4dff9-7795-48c5-b372-2f9b61f60156 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:51.141626 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 579/1154] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:51.146944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8578c-3ff9-4752-b724-3bc7cf4b0c0e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:51.147349 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8578c-3ff9-4752-b724-3bc7cf4b0c0e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:51.147492 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8578c-3ff9-4752-b724-3bc7cf4b0c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:51.167185 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7bc8578c-3ff9-4752-b724-3bc7cf4b0c0e None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:51.168381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7bc8578c-3ff9-4752-b724-3bc7cf4b0c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:51.208782 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:51.240039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 576/1155] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:51.510413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4dff9-7795-48c5-b372-2f9b61f60156 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:51.616524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4dff9-7795-48c5-b372-2f9b61f60156 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:51.650836 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cb231-ad59-455f-b1bf-c51f5f1c31b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:51.651467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cb231-ad59-455f-b1bf-c51f5f1c31b4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:51.652273 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cb231-ad59-455f-b1bf-c51f5f1c31b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:51.652548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cb231-ad59-455f-b1bf-c51f5f1c31b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:51.663237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9cb231-ad59-455f-b1bf-c51f5f1c31b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:47:51.668463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-6e9cb231-ad59-455f-b1bf-c51f5f1c31b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:47:51.668463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:51.668463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:51.668463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:51.668463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:51.668463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:51.668463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:51.668463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:51.668463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:51.668463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:51.668463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:51.668463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:51.668463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:47:51.668463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:47:51.668463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:47:51.668463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:47:51.669596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:47:51.669596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:47:51.669596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:47:51.669596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:47:51.669596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:51.669596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:51.669596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:51.669596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:51.669596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:51.669596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:51.669596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:51.669596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:51.669596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:51.669596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:51.669596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:51.669596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:51.670674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:51.670674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:51.670674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:51.670674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:51.670674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:47:51.670674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:47:51.670674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:47:51.670674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:47:51.670674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:47:51.670674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:47:51.670674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:47:51.670674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:47:51.670674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 580/1156] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:47:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:51.675919 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8578c-3ff9-4752-b724-3bc7cf4b0c0e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:51.775368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8578c-3ff9-4752-b724-3bc7cf4b0c0e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:51.817597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3068f-932d-4251-882d-75990e81c5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:51.818239 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3068f-932d-4251-882d-75990e81c5bb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:51.818519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3068f-932d-4251-882d-75990e81c5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:51.818792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3068f-932d-4251-882d-75990e81c5bb None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:51.819461 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd3068f-932d-4251-882d-75990e81c5bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:51.819761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd3068f-932d-4251-882d-75990e81c5bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:51.821126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd3068f-932d-4251-882d-75990e81c5bb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:51.829808 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 577/1157] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:47:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:51.835730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cb231-ad59-455f-b1bf-c51f5f1c31b4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:51.892133 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:51.937468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cb231-ad59-455f-b1bf-c51f5f1c31b4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:51.971542 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863399ee-d1e2-4c79-95bd-90c38e7deedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:51.972239 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863399ee-d1e2-4c79-95bd-90c38e7deedc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:51.972518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863399ee-d1e2-4c79-95bd-90c38e7deedc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:51.972792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863399ee-d1e2-4c79-95bd-90c38e7deedc None admin] PATH_INFO: `/v3/users/f5e8ee18498647e5b3ac9afa31d37453` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:51.973993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863399ee-d1e2-4c79-95bd-90c38e7deedc None admin] RBAC: Authorizing `identity:get_user(user_id=f5e8ee18498647e5b3ac9afa31d37453)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:51.975427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863399ee-d1e2-4c79-95bd-90c38e7deedc None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:51.977011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 581/1158] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:47:51 2022] GET /identity/v3/users/f5e8ee18498647e5b3ac9afa31d37453 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:51.983869 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3068f-932d-4251-882d-75990e81c5bb None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:52.018453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:52.086609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3068f-932d-4251-882d-75990e81c5bb None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:52.122220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55649f42-4f04-4062-bb5a-85ba1c0a8c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:52.122823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55649f42-4f04-4062-bb5a-85ba1c0a8c83 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:52.123104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55649f42-4f04-4062-bb5a-85ba1c0a8c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:52.123375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55649f42-4f04-4062-bb5a-85ba1c0a8c83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:52.189490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55649f42-4f04-4062-bb5a-85ba1c0a8c83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:52.190956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55649f42-4f04-4062-bb5a-85ba1c0a8c83 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:52.222316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:52.293634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 578/1159] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:47:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:47:52.299911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863399ee-d1e2-4c79-95bd-90c38e7deedc None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:52.400542 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863399ee-d1e2-4c79-95bd-90c38e7deedc None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:52.424715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:52.436203 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba0885-b137-4dff-9637-13eecc9cf96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:52.436841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba0885-b137-4dff-9637-13eecc9cf96b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:52.437128 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba0885-b137-4dff-9637-13eecc9cf96b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:52.437409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba0885-b137-4dff-9637-13eecc9cf96b None admin] PATH_INFO: `/v3/projects/2b25126932324dd6ad035447cd1553d6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:52.438621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba0885-b137-4dff-9637-13eecc9cf96b None admin] RBAC: Authorizing `identity:get_project(project_id=2b25126932324dd6ad035447cd1553d6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:52.440127 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba0885-b137-4dff-9637-13eecc9cf96b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:52.441775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 582/1160] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:47:52 2022] GET /identity/v3/projects/2b25126932324dd6ad035447cd1553d6 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:52.448004 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55649f42-4f04-4062-bb5a-85ba1c0a8c83 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:52.550464 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55649f42-4f04-4062-bb5a-85ba1c0a8c83 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:52.592755 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc88ce1-77f0-4834-9680-559a9c85e1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:52.593939 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc88ce1-77f0-4834-9680-559a9c85e1de None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:52.594528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc88ce1-77f0-4834-9680-559a9c85e1de None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:52.595070 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc88ce1-77f0-4834-9680-559a9c85e1de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:52.596450 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc88ce1-77f0-4834-9680-559a9c85e1de None admin] RBAC: Adding query filter params (role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.project.id=2b25126932324dd6ad035447cd1553d6, user.id=f5e8ee18498647e5b3ac9afa31d37453) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:52.597352 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc88ce1-77f0-4834-9680-559a9c85e1de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:52.600525 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc88ce1-77f0-4834-9680-559a9c85e1de None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:52.604044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:52.610712 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 579/1161] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:47:52 2022] GET /identity/v3/role_assignments?user.id=f5e8ee18498647e5b3ac9afa31d37453&scope.project.id=2b25126932324dd6ad035447cd1553d6&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:47:54.819884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86eedcb-97e7-48f8-ad83-f9a65f6324cf None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:54.820457 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86eedcb-97e7-48f8-ad83-f9a65f6324cf None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:54.820457 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86eedcb-97e7-48f8-ad83-f9a65f6324cf None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:54.821771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 583/1162] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:54.827143 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af67491-d886-41b3-b25c-4aa0ba6de4e7 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:54.827577 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af67491-d886-41b3-b25c-4aa0ba6de4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:54.827708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af67491-d886-41b3-b25c-4aa0ba6de4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:54.846313 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8af67491-d886-41b3-b25c-4aa0ba6de4e7 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:54.847576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8af67491-d886-41b3-b25c-4aa0ba6de4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:54.913023 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 580/1163] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:55.186163 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86eedcb-97e7-48f8-ad83-f9a65f6324cf None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:55.285644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86eedcb-97e7-48f8-ad83-f9a65f6324cf None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:55.324787 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb4b59-b6dc-4eea-a557-cef15851b353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:55.325397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb4b59-b6dc-4eea-a557-cef15851b353 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:55.325682 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb4b59-b6dc-4eea-a557-cef15851b353 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:55.325988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb4b59-b6dc-4eea-a557-cef15851b353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:55.386950 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:55.396166 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb4b59-b6dc-4eea-a557-cef15851b353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:55.411025 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb4b59-b6dc-4eea-a557-cef15851b353 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:55.498922 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 584/1164] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:47:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:47:55.504851 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af67491-d886-41b3-b25c-4aa0ba6de4e7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:55.603436 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af67491-d886-41b3-b25c-4aa0ba6de4e7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:55.629739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:55.637531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d937f2f-c2a1-4d9c-b2ce-98b2813f045f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:55.645888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d937f2f-c2a1-4d9c-b2ce-98b2813f045f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:55.645888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d937f2f-c2a1-4d9c-b2ce-98b2813f045f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:55.645888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d937f2f-c2a1-4d9c-b2ce-98b2813f045f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:55.645888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d937f2f-c2a1-4d9c-b2ce-98b2813f045f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:55.647153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d937f2f-c2a1-4d9c-b2ce-98b2813f045f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:55.659691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-6d937f2f-c2a1-4d9c-b2ce-98b2813f045f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 14 11:47:55.659691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:55.659691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 14 11:47:55.659691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 14 11:47:55.659691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:47:55.659691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:47:55.659691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:47:55.659691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:47:55.659691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 14 11:47:55.659691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:55.659691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:47:55.659691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:55.659691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:47:55.659691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:47:55.659691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:47:55.659691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:47:55.659691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:47:55.661749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:47:55.661749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:47:55.661749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:47:55.661749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:47:55.661749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:47:55.661749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:47:55.661749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:47:55.661749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:47:55.661749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:47:55.661749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:47:55.661749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:47:55.661749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:47:55.661749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:47:55.661749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:47:55.661749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:47:55.661749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:47:55.663806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:47:55.663806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:47:55.663806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:47:55.663806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:47:55.663806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:47:55.663806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:47:55.663806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:47:55.663806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:47:55.663806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:47:55.663806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:47:55.663806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 14 11:47:55.663806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:47:55.663806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 14 11:47:55.663806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:47:55.663806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 581/1165] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 14 11:47:55 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:47:55.667000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb4b59-b6dc-4eea-a557-cef15851b353 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:55.761423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:47:55.770192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb4b59-b6dc-4eea-a557-cef15851b353 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:55.804473 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2f7e1-3730-4697-adf4-92ee3b990b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:55.805085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2f7e1-3730-4697-adf4-92ee3b990b4c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:55.805351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2f7e1-3730-4697-adf4-92ee3b990b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:55.805622 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2f7e1-3730-4697-adf4-92ee3b990b4c None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:55.806287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a2f7e1-3730-4697-adf4-92ee3b990b4c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:47:55.806588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a2f7e1-3730-4697-adf4-92ee3b990b4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:55.807970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a2f7e1-3730-4697-adf4-92ee3b990b4c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:55.814799 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 585/1166] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Sep 14 11:47:55 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:47:57.928421 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf72ba-3912-4b78-8a51-63f3fc2859c8 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:57.928975 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf72ba-3912-4b78-8a51-63f3fc2859c8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:57.928975 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf72ba-3912-4b78-8a51-63f3fc2859c8 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:57.930373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 582/1167] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:47:57.935948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7056ff-9d3e-4784-becd-ed64e7bff391 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:57.936397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7056ff-9d3e-4784-becd-ed64e7bff391 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:57.936501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7056ff-9d3e-4784-becd-ed64e7bff391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:57.955311 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ec7056ff-9d3e-4784-becd-ed64e7bff391 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:47:57.956534 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ec7056ff-9d3e-4784-becd-ed64e7bff391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:47:58.023427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 586/1168] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:47:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:47:58.290702 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf72ba-3912-4b78-8a51-63f3fc2859c8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:47:58.397274 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf72ba-3912-4b78-8a51-63f3fc2859c8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:47:58.432283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2636d-f74b-4985-b97b-0de958562082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:47:58.433439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2636d-f74b-4985-b97b-0de958562082 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:47:58.433716 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2636d-f74b-4985-b97b-0de958562082 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:47:58.434023 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2636d-f74b-4985-b97b-0de958562082 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:47:58.434709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2636d-f74b-4985-b97b-0de958562082 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:47:58.436064 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2636d-f74b-4985-b97b-0de958562082 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:47:58.505679 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 583/1169] 10.222.0.62 () {60 vars in 1138 bytes} [Wed Sep 14 11:47:58 2022] POST /identity/v3/domains => generated 267 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:48:00.704512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c78c7b-58a1-4c16-8b21-d5c1048d720d None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:00.705067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c78c7b-58a1-4c16-8b21-d5c1048d720d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:00.705067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c78c7b-58a1-4c16-8b21-d5c1048d720d None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:00.706459 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 587/1170] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:48:00.711751 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9381389-abc6-42a7-9306-2b225268d5c5 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:00.712152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9381389-abc6-42a7-9306-2b225268d5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:00.712303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9381389-abc6-42a7-9306-2b225268d5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:00.731109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f9381389-abc6-42a7-9306-2b225268d5c5 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:48:00.732270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f9381389-abc6-42a7-9306-2b225268d5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:48:00.773167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:00.795789 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 584/1171] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:48:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:48:01.066106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c78c7b-58a1-4c16-8b21-d5c1048d720d None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:01.153535 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:01.169765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c78c7b-58a1-4c16-8b21-d5c1048d720d None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:01.205323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc07dca-370e-4856-93a3-0de7229b464c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:01.205979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc07dca-370e-4856-93a3-0de7229b464c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:01.206266 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc07dca-370e-4856-93a3-0de7229b464c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:01.206546 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc07dca-370e-4856-93a3-0de7229b464c None admin] PATH_INFO: `/v3/domains/cc7c5b6f600a4e13b930fe76ea87347c` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:01.217082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc07dca-370e-4856-93a3-0de7229b464c None admin] RBAC: Authorizing `identity:get_domain(domain_id=cc7c5b6f600a4e13b930fe76ea87347c)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:48:01.218558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc07dca-370e-4856-93a3-0de7229b464c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:48:01.220194 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 588/1172] 10.222.0.62 () {56 vars in 1183 bytes} [Wed Sep 14 11:48:01 2022] GET /identity/v3/domains/cc7c5b6f600a4e13b930fe76ea87347c => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:48:01.226120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9381389-abc6-42a7-9306-2b225268d5c5 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:01.327974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9381389-abc6-42a7-9306-2b225268d5c5 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:01.361634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee93cd7-af57-45ad-a5d0-e39c13c246ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:01.362272 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee93cd7-af57-45ad-a5d0-e39c13c246ea None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:01.362558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee93cd7-af57-45ad-a5d0-e39c13c246ea None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:01.362834 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee93cd7-af57-45ad-a5d0-e39c13c246ea None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:01.363482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee93cd7-af57-45ad-a5d0-e39c13c246ea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:48:01.364907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee93cd7-af57-45ad-a5d0-e39c13c246ea None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:48:01.472116 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 585/1173] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 14 11:48:01 2022] POST /identity/v3/projects => generated 361 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:48:03.700326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de446d22-956e-42f4-95b7-b509afd4be64 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:03.700861 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de446d22-956e-42f4-95b7-b509afd4be64 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:03.700861 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de446d22-956e-42f4-95b7-b509afd4be64 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:03.702250 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 589/1174] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:48:03.707680 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96e798-9cdc-4794-bacb-12d3522556ae None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:03.708115 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96e798-9cdc-4794-bacb-12d3522556ae None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:03.708246 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96e798-9cdc-4794-bacb-12d3522556ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:03.726850 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7b96e798-9cdc-4794-bacb-12d3522556ae None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:48:03.728048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7b96e798-9cdc-4794-bacb-12d3522556ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:48:03.751877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:03.794065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 586/1175] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:48:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:48:04.075759 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de446d22-956e-42f4-95b7-b509afd4be64 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:04.188636 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de446d22-956e-42f4-95b7-b509afd4be64 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:04.223402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd633a-8b38-450b-b389-135153bfbcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:04.224033 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd633a-8b38-450b-b389-135153bfbcbe None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:04.224319 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd633a-8b38-450b-b389-135153bfbcbe None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:04.224595 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd633a-8b38-450b-b389-135153bfbcbe None admin] PATH_INFO: `/v3/domains/cc7c5b6f600a4e13b930fe76ea87347c` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:04.225728 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd633a-8b38-450b-b389-135153bfbcbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=cc7c5b6f600a4e13b930fe76ea87347c)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:48:04.227207 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd633a-8b38-450b-b389-135153bfbcbe None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:48:04.228796 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 590/1176] 10.222.0.62 () {56 vars in 1183 bytes} [Wed Sep 14 11:48:04 2022] GET /identity/v3/domains/cc7c5b6f600a4e13b930fe76ea87347c => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:48:04.234590 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96e798-9cdc-4794-bacb-12d3522556ae None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:04.330688 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96e798-9cdc-4794-bacb-12d3522556ae None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:04.363811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9330cec-85ad-45a5-986d-1275a22a9289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:04.364422 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9330cec-85ad-45a5-986d-1275a22a9289 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:04.364705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9330cec-85ad-45a5-986d-1275a22a9289 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:04.364981 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9330cec-85ad-45a5-986d-1275a22a9289 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:04.365638 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9330cec-85ad-45a5-986d-1275a22a9289 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:48:04.367112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9330cec-85ad-45a5-986d-1275a22a9289 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:48:04.430016 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 587/1177] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 14 11:48:04 2022] POST /identity/v3/users => generated 311 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:48:06.669519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573208a4-f5a8-4553-a8f3-986180d5152f None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:06.670120 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573208a4-f5a8-4553-a8f3-986180d5152f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:06.670120 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573208a4-f5a8-4553-a8f3-986180d5152f None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:06.671463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 591/1178] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:48:06.677088 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc2b61-6e65-48aa-ab42-9becb2d4c420 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:06.677522 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc2b61-6e65-48aa-ab42-9becb2d4c420 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:06.677722 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc2b61-6e65-48aa-ab42-9becb2d4c420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:06.696033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-16cc2b61-6e65-48aa-ab42-9becb2d4c420 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:48:06.697221 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-16cc2b61-6e65-48aa-ab42-9becb2d4c420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:48:06.708449 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:06.766042 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 588/1179] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:48:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:48:07.039143 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573208a4-f5a8-4553-a8f3-986180d5152f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:07.058292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:07.149150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573208a4-f5a8-4553-a8f3-986180d5152f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:07.183967 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967562e1-79c4-44be-81a2-d4360f49aa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:07.184596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967562e1-79c4-44be-81a2-d4360f49aa9f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:07.184882 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967562e1-79c4-44be-81a2-d4360f49aa9f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:07.185160 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967562e1-79c4-44be-81a2-d4360f49aa9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:07.196738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967562e1-79c4-44be-81a2-d4360f49aa9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:48:07.202186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-967562e1-79c4-44be-81a2-d4360f49aa9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:48:07.202186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:48:07.202186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:48:07.202186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:48:07.202186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:48:07.202186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:48:07.202186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:48:07.202186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:48:07.202186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:48:07.202186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:48:07.202186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:48:07.202186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:48:07.202186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:48:07.202186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:48:07.202186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:48:07.202186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:48:07.203377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:48:07.203377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:48:07.203377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:48:07.203377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:48:07.203377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:48:07.203377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:48:07.203377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:48:07.203377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:48:07.203377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:48:07.203377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:48:07.203377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:48:07.203377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:48:07.203377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:48:07.203377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:48:07.203377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:48:07.203377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:48:07.204241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:48:07.204241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:48:07.204241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:48:07.204241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:48:07.204241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:48:07.204241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:48:07.204241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:48:07.204241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:48:07.204241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:48:07.204241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:48:07.204241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:48:07.204241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:48:07.204241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 592/1180] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:48:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:48:07.209591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc2b61-6e65-48aa-ab42-9becb2d4c420 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:07.311294 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc2b61-6e65-48aa-ab42-9becb2d4c420 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:07.344189 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c30cb7-5e55-420d-a53a-b22d6e7ca6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:07.344792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c30cb7-5e55-420d-a53a-b22d6e7ca6ea None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:07.345077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c30cb7-5e55-420d-a53a-b22d6e7ca6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:07.345349 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c30cb7-5e55-420d-a53a-b22d6e7ca6ea None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:07.346035 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c30cb7-5e55-420d-a53a-b22d6e7ca6ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:48:07.346340 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c30cb7-5e55-420d-a53a-b22d6e7ca6ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:48:07.347704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c30cb7-5e55-420d-a53a-b22d6e7ca6ea None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:48:07.356435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 589/1181] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:48:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:48:07.362590 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967562e1-79c4-44be-81a2-d4360f49aa9f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:07.436843 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:07.460542 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967562e1-79c4-44be-81a2-d4360f49aa9f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:07.493692 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca590c-2f0a-4e0b-8753-6bb2157bc467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:07.494343 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca590c-2f0a-4e0b-8753-6bb2157bc467 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:07.494652 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca590c-2f0a-4e0b-8753-6bb2157bc467 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:07.494930 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca590c-2f0a-4e0b-8753-6bb2157bc467 None admin] PATH_INFO: `/v3/users/0cee38af94ed457b861fd1661867ab1a` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:07.507630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca590c-2f0a-4e0b-8753-6bb2157bc467 None admin] RBAC: Authorizing `identity:get_user(user_id=0cee38af94ed457b861fd1661867ab1a)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:48:07.509069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca590c-2f0a-4e0b-8753-6bb2157bc467 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:48:07.510711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 593/1182] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:48:07 2022] GET /identity/v3/users/0cee38af94ed457b861fd1661867ab1a => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:48:07.516560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c30cb7-5e55-420d-a53a-b22d6e7ca6ea None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:07.528118 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:07.614077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c30cb7-5e55-420d-a53a-b22d6e7ca6ea None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:07.646916 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd3e72-b08f-4644-9e09-e5a8e8c9e84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:07.647512 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd3e72-b08f-4644-9e09-e5a8e8c9e84a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:07.647794 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd3e72-b08f-4644-9e09-e5a8e8c9e84a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:07.648069 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd3e72-b08f-4644-9e09-e5a8e8c9e84a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:07.712410 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd3e72-b08f-4644-9e09-e5a8e8c9e84a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:48:07.713834 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd3e72-b08f-4644-9e09-e5a8e8c9e84a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:48:07.723069 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:07.809809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 590/1183] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:48:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:48:07.815794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca590c-2f0a-4e0b-8753-6bb2157bc467 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:07.912451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca590c-2f0a-4e0b-8753-6bb2157bc467 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:07.940276 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:07.944954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb504b7-39e1-45ad-837d-1bfacdf3a496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:07.945571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb504b7-39e1-45ad-837d-1bfacdf3a496 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:07.945850 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb504b7-39e1-45ad-837d-1bfacdf3a496 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:07.946155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb504b7-39e1-45ad-837d-1bfacdf3a496 None admin] PATH_INFO: `/v3/projects/4822e7b73ffb4a908d0c82b5e968965e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:07.947247 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb504b7-39e1-45ad-837d-1bfacdf3a496 None admin] RBAC: Authorizing `identity:get_project(project_id=4822e7b73ffb4a908d0c82b5e968965e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:48:07.949192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb504b7-39e1-45ad-837d-1bfacdf3a496 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:48:07.950854 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 594/1184] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:48:07 2022] GET /identity/v3/projects/4822e7b73ffb4a908d0c82b5e968965e => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:48:07.956986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd3e72-b08f-4644-9e09-e5a8e8c9e84a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:08.054174 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd3e72-b08f-4644-9e09-e5a8e8c9e84a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:08.063075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:08.087287 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec202abc-603d-47ba-9c20-ab02225c1f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:08.087882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec202abc-603d-47ba-9c20-ab02225c1f34 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:08.088164 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec202abc-603d-47ba-9c20-ab02225c1f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:08.089083 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec202abc-603d-47ba-9c20-ab02225c1f34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:08.089805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec202abc-603d-47ba-9c20-ab02225c1f34 None admin] RBAC: Adding query filter params (role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.project.id=4822e7b73ffb4a908d0c82b5e968965e, user.id=0cee38af94ed457b861fd1661867ab1a) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:48:08.090147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec202abc-603d-47ba-9c20-ab02225c1f34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:48:08.091635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec202abc-603d-47ba-9c20-ab02225c1f34 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:48:08.097446 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 591/1185] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:48:07 2022] GET /identity/v3/role_assignments?user.id=0cee38af94ed457b861fd1661867ab1a&scope.project.id=4822e7b73ffb4a908d0c82b5e968965e&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:48:10.192049 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d085765-6751-4c15-9662-63df425769b8 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:10.192609 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d085765-6751-4c15-9662-63df425769b8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:10.192609 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d085765-6751-4c15-9662-63df425769b8 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:10.193978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 595/1186] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:48:10.199852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d60fb3-49e2-402c-bc66-2b0be7e4cbac None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:10.200272 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d60fb3-49e2-402c-bc66-2b0be7e4cbac None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:10.200409 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d60fb3-49e2-402c-bc66-2b0be7e4cbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:10.218917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-b5d60fb3-49e2-402c-bc66-2b0be7e4cbac None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:48:10.220111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b5d60fb3-49e2-402c-bc66-2b0be7e4cbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:48:10.286036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 592/1187] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:48:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:48:10.553413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d085765-6751-4c15-9662-63df425769b8 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:10.659439 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d085765-6751-4c15-9662-63df425769b8 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:10.694774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3935d0-276e-4f5d-b5dc-6136c57185bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:10.695398 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3935d0-276e-4f5d-b5dc-6136c57185bb None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:10.695680 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3935d0-276e-4f5d-b5dc-6136c57185bb None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:10.695955 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3935d0-276e-4f5d-b5dc-6136c57185bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:10.706451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3935d0-276e-4f5d-b5dc-6136c57185bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:48:10.711295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-7d3935d0-276e-4f5d-b5dc-6136c57185bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:48:10.711295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:48:10.711295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:48:10.711295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:48:10.711295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:48:10.711295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:48:10.711295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:48:10.711295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:48:10.711295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:48:10.711295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:48:10.711295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:48:10.711295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:48:10.711295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:48:10.711295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:48:10.711295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:48:10.711295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:48:10.712410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:48:10.712410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:48:10.712410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:48:10.712410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:48:10.712410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:48:10.712410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:48:10.712410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:48:10.712410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:48:10.712410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:48:10.712410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:48:10.712410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:48:10.712410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:48:10.712410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:48:10.712410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:48:10.712410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:48:10.712410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:48:10.713490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:48:10.713490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:48:10.713490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:48:10.713490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:48:10.713490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:48:10.713490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:48:10.713490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:48:10.713490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:48:10.713490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:48:10.713490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:48:10.713490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:48:10.713490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:48:10.713490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 596/1188] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:48:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:48:10.718538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d60fb3-49e2-402c-bc66-2b0be7e4cbac None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:10.821071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d60fb3-49e2-402c-bc66-2b0be7e4cbac None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:10.823112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:10.857189 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc86470-ac06-4339-acb1-33fac17298c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:10.858163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc86470-ac06-4339-acb1-33fac17298c5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:10.858631 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc86470-ac06-4339-acb1-33fac17298c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:10.859220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc86470-ac06-4339-acb1-33fac17298c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:10.859893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc86470-ac06-4339-acb1-33fac17298c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:48:10.860208 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc86470-ac06-4339-acb1-33fac17298c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:48:10.861681 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc86470-ac06-4339-acb1-33fac17298c5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:48:10.871360 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 593/1189] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:48:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:48:10.877371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3935d0-276e-4f5d-b5dc-6136c57185bb None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:10.944038 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:10.987188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3935d0-276e-4f5d-b5dc-6136c57185bb None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:11.030806 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e7861-22a0-4198-aac0-2f89d810a344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:11.031436 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e7861-22a0-4198-aac0-2f89d810a344 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:11.031715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e7861-22a0-4198-aac0-2f89d810a344 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:11.031986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e7861-22a0-4198-aac0-2f89d810a344 None admin] PATH_INFO: `/v3/users/0cee38af94ed457b861fd1661867ab1a` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:11.033143 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e7861-22a0-4198-aac0-2f89d810a344 None admin] RBAC: Authorizing `identity:get_user(user_id=0cee38af94ed457b861fd1661867ab1a)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:48:11.034631 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e7861-22a0-4198-aac0-2f89d810a344 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:48:11.036219 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 597/1190] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:48:10 2022] GET /identity/v3/users/0cee38af94ed457b861fd1661867ab1a => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:48:11.042759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc86470-ac06-4339-acb1-33fac17298c5 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:11.145193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc86470-ac06-4339-acb1-33fac17298c5 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:11.178712 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52297b43-ba35-4a8d-9543-9fe504c95f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:11.179340 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52297b43-ba35-4a8d-9543-9fe504c95f1b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:11.179614 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52297b43-ba35-4a8d-9543-9fe504c95f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:11.179881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52297b43-ba35-4a8d-9543-9fe504c95f1b None admin] PATH_INFO: `/v3/projects/4822e7b73ffb4a908d0c82b5e968965e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:11.180980 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52297b43-ba35-4a8d-9543-9fe504c95f1b None admin] RBAC: Authorizing `identity:get_project(project_id=4822e7b73ffb4a908d0c82b5e968965e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:48:11.183168 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52297b43-ba35-4a8d-9543-9fe504c95f1b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:48:11.184782 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 594/1191] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:48:11 2022] GET /identity/v3/projects/4822e7b73ffb4a908d0c82b5e968965e => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:48:11.190835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e7861-22a0-4198-aac0-2f89d810a344 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:11.290349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e7861-22a0-4198-aac0-2f89d810a344 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:11.318348 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:11.330043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bf731-ab0c-4e47-b2e0-737707b29d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:11.330714 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bf731-ab0c-4e47-b2e0-737707b29d86 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:11.330997 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bf731-ab0c-4e47-b2e0-737707b29d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:11.331508 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bf731-ab0c-4e47-b2e0-737707b29d86 None admin] PATH_INFO: `/v3/projects/4822e7b73ffb4a908d0c82b5e968965e/users/0cee38af94ed457b861fd1661867ab1a/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:11.333363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bf731-ab0c-4e47-b2e0-737707b29d86 None admin] RBAC: Authorizing `identity:create_grant(project_id=4822e7b73ffb4a908d0c82b5e968965e, user_id=0cee38af94ed457b861fd1661867ab1a, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:48:11.334839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bf731-ab0c-4e47-b2e0-737707b29d86 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:48:11.401959 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 598/1192] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:48:11 2022] PUT /identity/v3/projects/4822e7b73ffb4a908d0c82b5e968965e/users/0cee38af94ed457b861fd1661867ab1a/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:48:13.551135 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224774e-ff31-456f-8ee7-2a7038a606f4 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:13.551670 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224774e-ff31-456f-8ee7-2a7038a606f4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:13.551670 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224774e-ff31-456f-8ee7-2a7038a606f4 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:13.553010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 595/1193] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:48:13.558789 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c3d6c-be35-492f-86cd-f4e780310fc2 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:13.559224 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c3d6c-be35-492f-86cd-f4e780310fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:13.559350 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c3d6c-be35-492f-86cd-f4e780310fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:13.587901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-7b7c3d6c-be35-492f-86cd-f4e780310fc2 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:48:13.589128 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7b7c3d6c-be35-492f-86cd-f4e780310fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:48:13.679494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 599/1194] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:48:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:48:13.948907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224774e-ff31-456f-8ee7-2a7038a606f4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:13.980794 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:14.050834 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224774e-ff31-456f-8ee7-2a7038a606f4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:14.083887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2cdb1e-3795-45bb-b128-941f83b19b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:14.084440 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cdb1e-3795-45bb-b128-941f83b19b22 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:14.084726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cdb1e-3795-45bb-b128-941f83b19b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:14.085000 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cdb1e-3795-45bb-b128-941f83b19b22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:14.095723 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2cdb1e-3795-45bb-b128-941f83b19b22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:48:14.100537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-4f2cdb1e-3795-45bb-b128-941f83b19b22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:48:14.100537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:48:14.100537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:48:14.100537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:48:14.100537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:48:14.100537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:48:14.100537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:48:14.100537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:48:14.100537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:48:14.100537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:48:14.100537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:48:14.100537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:48:14.100537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:48:14.100537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:48:14.100537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:48:14.100537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:48:14.101641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:48:14.101641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:48:14.101641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:48:14.101641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:48:14.101641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:48:14.101641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:48:14.101641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:48:14.101641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:48:14.101641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:48:14.101641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:48:14.101641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:48:14.101641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:48:14.101641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:48:14.101641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:48:14.101641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:48:14.101641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:48:14.102729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:48:14.102729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:48:14.102729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:48:14.102729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:48:14.102729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:48:14.102729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:48:14.102729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:48:14.102729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:48:14.102729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:48:14.102729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:48:14.102729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:48:14.102729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:48:14.102729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 596/1195] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:48:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:48:14.107739 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c3d6c-be35-492f-86cd-f4e780310fc2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:14.210093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c3d6c-be35-492f-86cd-f4e780310fc2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:14.227216 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:14.243711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40910cb-2781-429c-96c5-0b037196f967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:14.244786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40910cb-2781-429c-96c5-0b037196f967 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:14.245077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40910cb-2781-429c-96c5-0b037196f967 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:14.245430 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40910cb-2781-429c-96c5-0b037196f967 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:14.246114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40910cb-2781-429c-96c5-0b037196f967 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:48:14.246417 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40910cb-2781-429c-96c5-0b037196f967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:48:14.247772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40910cb-2781-429c-96c5-0b037196f967 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:48:14.255612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 600/1196] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:48:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:48:14.262206 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2cdb1e-3795-45bb-b128-941f83b19b22 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:14.355778 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:14.364137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2cdb1e-3795-45bb-b128-941f83b19b22 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:14.397724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d238b-dad7-4d24-94f5-a2397ffbec8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:14.398368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d238b-dad7-4d24-94f5-a2397ffbec8f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:14.398645 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d238b-dad7-4d24-94f5-a2397ffbec8f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:14.398915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d238b-dad7-4d24-94f5-a2397ffbec8f None admin] PATH_INFO: `/v3/users/0cee38af94ed457b861fd1661867ab1a` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:14.400015 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d238b-dad7-4d24-94f5-a2397ffbec8f None admin] RBAC: Authorizing `identity:get_user(user_id=0cee38af94ed457b861fd1661867ab1a)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:48:14.401455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d238b-dad7-4d24-94f5-a2397ffbec8f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:48:14.403060 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 597/1197] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 14 11:48:14 2022] GET /identity/v3/users/0cee38af94ed457b861fd1661867ab1a => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:48:14.408978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40910cb-2781-429c-96c5-0b037196f967 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:14.508290 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40910cb-2781-429c-96c5-0b037196f967 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:14.542953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33753723-9406-4a46-9f8e-fbc1a72745f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:14.543547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33753723-9406-4a46-9f8e-fbc1a72745f4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:14.543835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33753723-9406-4a46-9f8e-fbc1a72745f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:14.544114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33753723-9406-4a46-9f8e-fbc1a72745f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:14.591844 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:14.616279 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33753723-9406-4a46-9f8e-fbc1a72745f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:48:14.616279 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33753723-9406-4a46-9f8e-fbc1a72745f4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:48:14.709509 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 601/1198] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:48:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:48:14.716147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d238b-dad7-4d24-94f5-a2397ffbec8f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:14.816608 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d238b-dad7-4d24-94f5-a2397ffbec8f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:14.851517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44666f-a22f-432d-987e-4c98a4bcda6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:14.852146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44666f-a22f-432d-987e-4c98a4bcda6e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:14.852432 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44666f-a22f-432d-987e-4c98a4bcda6e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:14.852710 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44666f-a22f-432d-987e-4c98a4bcda6e None admin] PATH_INFO: `/v3/projects/4822e7b73ffb4a908d0c82b5e968965e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:14.853881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44666f-a22f-432d-987e-4c98a4bcda6e None admin] RBAC: Authorizing `identity:get_project(project_id=4822e7b73ffb4a908d0c82b5e968965e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:48:14.855331 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44666f-a22f-432d-987e-4c98a4bcda6e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:48:14.856950 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 598/1199] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 14 11:48:14 2022] GET /identity/v3/projects/4822e7b73ffb4a908d0c82b5e968965e => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:48:14.863113 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33753723-9406-4a46-9f8e-fbc1a72745f4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:48:14.943250 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:14.962620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33753723-9406-4a46-9f8e-fbc1a72745f4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:48:14.996359 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cba8ed-f507-4e10-8d53-96245414237b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:48:14.997108 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cba8ed-f507-4e10-8d53-96245414237b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:48:14.997762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cba8ed-f507-4e10-8d53-96245414237b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:48:14.997762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cba8ed-f507-4e10-8d53-96245414237b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:48:14.998352 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cba8ed-f507-4e10-8d53-96245414237b None admin] RBAC: Adding query filter params (role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.project.id=4822e7b73ffb4a908d0c82b5e968965e, user.id=0cee38af94ed457b861fd1661867ab1a) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:48:14.998687 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cba8ed-f507-4e10-8d53-96245414237b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:48:15.000126 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cba8ed-f507-4e10-8d53-96245414237b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:48:15.006172 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 602/1200] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:48:14 2022] GET /identity/v3/role_assignments?user.id=0cee38af94ed457b861fd1661867ab1a&scope.project.id=4822e7b73ffb4a908d0c82b5e968965e&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:48:24.847296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=86 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:48:25.003268 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=28 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:29.488901 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825d2d1-1b07-4398-a67f-52854576b964 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:29.489490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825d2d1-1b07-4398-a67f-52854576b964 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:29.489490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825d2d1-1b07-4398-a67f-52854576b964 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:29.490885 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 599/1201] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:49:29.496533 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527efe15-9b41-4754-88c2-e91241273af2 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:29.497024 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527efe15-9b41-4754-88c2-e91241273af2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:29.497128 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527efe15-9b41-4754-88c2-e91241273af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:29.518032 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-527efe15-9b41-4754-88c2-e91241273af2 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:49:29.525562 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-527efe15-9b41-4754-88c2-e91241273af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:49:29.593260 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 603/1202] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:49:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:49:29.867526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825d2d1-1b07-4398-a67f-52854576b964 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:29.976016 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825d2d1-1b07-4398-a67f-52854576b964 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:30.010127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c72f4-c069-4494-a4b3-191315cad57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:30.010816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c72f4-c069-4494-a4b3-191315cad57c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:30.011068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c72f4-c069-4494-a4b3-191315cad57c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:30.011341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c72f4-c069-4494-a4b3-191315cad57c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:30.023483 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c72f4-c069-4494-a4b3-191315cad57c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:30.024987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c72f4-c069-4494-a4b3-191315cad57c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:30.041108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-228c72f4-c069-4494-a4b3-191315cad57c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:30.041108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:30.041108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:30.041108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:30.041108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:30.041108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:30.041108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:30.041108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:30.041108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:30.041108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:30.041108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:30.041108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:30.041108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:49:30.041108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:49:30.041108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:49:30.041108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:49:30.042974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:30.042974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:30.042974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:30.042974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:30.042974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:30.042974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:30.042974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:30.042974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:30.042974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:30.042974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:30.042974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:30.042974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:30.042974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:30.042974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:30.042974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:30.042974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:30.042974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:49:30.044369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:49:30.044369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:30.044369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:49:30.044369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 600/1203] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:30.049184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527efe15-9b41-4754-88c2-e91241273af2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:30.168585 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527efe15-9b41-4754-88c2-e91241273af2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:30.202180 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:30.208754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f085b9e-4d45-4371-b4f1-61040c3f320d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:30.209426 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f085b9e-4d45-4371-b4f1-61040c3f320d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:30.209773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f085b9e-4d45-4371-b4f1-61040c3f320d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:30.210128 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f085b9e-4d45-4371-b4f1-61040c3f320d None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:30.210858 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f085b9e-4d45-4371-b4f1-61040c3f320d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:30.211209 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f085b9e-4d45-4371-b4f1-61040c3f320d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:30.212686 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f085b9e-4d45-4371-b4f1-61040c3f320d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:30.225176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 604/1204] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:30.231359 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c72f4-c069-4494-a4b3-191315cad57c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:30.290072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:30.343348 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c72f4-c069-4494-a4b3-191315cad57c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:30.378568 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c190ca3-6685-48b1-acf6-6634271fdd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:30.379181 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c190ca3-6685-48b1-acf6-6634271fdd55 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:30.379463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c190ca3-6685-48b1-acf6-6634271fdd55 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:30.379739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c190ca3-6685-48b1-acf6-6634271fdd55 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:30.380396 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c190ca3-6685-48b1-acf6-6634271fdd55 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:30.381926 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c190ca3-6685-48b1-acf6-6634271fdd55 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:30.486897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 601/1205] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 14 11:49:30 2022] POST /identity/v3/users => generated 251 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:49:32.724088 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8a3d-e961-446d-aef1-e8bf9e86b31d None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:32.724689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8a3d-e961-446d-aef1-e8bf9e86b31d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:32.724689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8a3d-e961-446d-aef1-e8bf9e86b31d None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:32.726095 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 605/1206] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:49:32.731681 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb8d6e-d229-454a-a4db-92b15c21490d None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:32.732151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb8d6e-d229-454a-a4db-92b15c21490d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:32.732291 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb8d6e-d229-454a-a4db-92b15c21490d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:32.751065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f9eb8d6e-d229-454a-a4db-92b15c21490d None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:49:32.752270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f9eb8d6e-d229-454a-a4db-92b15c21490d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:49:32.819739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 602/1207] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:49:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:49:33.094034 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc8a3d-e961-446d-aef1-e8bf9e86b31d None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:33.201242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc8a3d-e961-446d-aef1-e8bf9e86b31d None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:33.238371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99410f00-a52d-4b7e-9e0d-ebc88a17828c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:33.239009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99410f00-a52d-4b7e-9e0d-ebc88a17828c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:33.239294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99410f00-a52d-4b7e-9e0d-ebc88a17828c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:33.239574 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99410f00-a52d-4b7e-9e0d-ebc88a17828c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:33.250541 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99410f00-a52d-4b7e-9e0d-ebc88a17828c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:49:33.255512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-99410f00-a52d-4b7e-9e0d-ebc88a17828c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:49:33.255512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:33.255512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:33.255512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:33.255512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:33.255512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:33.255512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:33.255512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:33.255512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:33.255512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:33.255512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:33.255512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:33.255512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:49:33.255512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:49:33.255512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:49:33.255512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:49:33.257012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:49:33.257012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:49:33.257012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:49:33.257012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:49:33.257012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:33.257012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:33.257012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:33.257012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:33.257012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:33.257012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:33.257012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:33.257012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:33.257012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:33.257012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:33.257012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:33.257012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:33.258015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:33.258015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:33.258015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:33.258015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:33.258015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:49:33.258015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:49:33.258015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:49:33.258015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:49:33.258015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:49:33.258015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:49:33.258015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:49:33.258015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:49:33.258015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 606/1208] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:49:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:33.262785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb8d6e-d229-454a-a4db-92b15c21490d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:33.309187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:33.368728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb8d6e-d229-454a-a4db-92b15c21490d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:33.404875 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924fdadb-f083-420d-851d-9a665552d246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:33.405546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fdadb-f083-420d-851d-9a665552d246 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:33.405835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fdadb-f083-420d-851d-9a665552d246 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:33.406141 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fdadb-f083-420d-851d-9a665552d246 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:33.406811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924fdadb-f083-420d-851d-9a665552d246 None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:33.407119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924fdadb-f083-420d-851d-9a665552d246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:33.408498 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924fdadb-f083-420d-851d-9a665552d246 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:33.416994 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 603/1209] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:49:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:33.423187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99410f00-a52d-4b7e-9e0d-ebc88a17828c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:33.473495 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:33.523761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99410f00-a52d-4b7e-9e0d-ebc88a17828c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:33.558634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57824d92-c21d-485d-a73e-23adb6023c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:33.559268 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57824d92-c21d-485d-a73e-23adb6023c67 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:33.560183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57824d92-c21d-485d-a73e-23adb6023c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:33.560462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57824d92-c21d-485d-a73e-23adb6023c67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:33.571644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57824d92-c21d-485d-a73e-23adb6023c67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:33.573083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57824d92-c21d-485d-a73e-23adb6023c67 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:33.588135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-57824d92-c21d-485d-a73e-23adb6023c67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:33.588135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:33.588135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:33.588135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:33.588135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:33.588135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:33.588135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:33.588135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:33.588135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:33.588135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:33.588135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:33.588135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:33.588135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:49:33.588135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:49:33.588135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:49:33.588135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:49:33.589077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:33.589077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:33.589077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:33.589077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:33.589077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:33.589077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:33.589077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:33.589077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:33.589077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:33.589077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:33.589077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:33.589077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:33.589077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:33.589077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:33.589077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:33.589077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:33.589077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:49:33.590042 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:49:33.590042 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:33.590042 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:49:33.590042 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 607/1210] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:33.595278 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924fdadb-f083-420d-851d-9a665552d246 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:33.693999 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:33.696817 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924fdadb-f083-420d-851d-9a665552d246 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:33.729081 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6929bab-8c51-4ce4-9971-2fdd47470d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:33.729731 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6929bab-8c51-4ce4-9971-2fdd47470d5b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:33.729997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6929bab-8c51-4ce4-9971-2fdd47470d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:33.730273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6929bab-8c51-4ce4-9971-2fdd47470d5b None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:33.730916 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6929bab-8c51-4ce4-9971-2fdd47470d5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:33.731704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6929bab-8c51-4ce4-9971-2fdd47470d5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:33.733077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6929bab-8c51-4ce4-9971-2fdd47470d5b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:33.744498 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 604/1211] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:33.750902 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57824d92-c21d-485d-a73e-23adb6023c67 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:33.853115 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57824d92-c21d-485d-a73e-23adb6023c67 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:33.854755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:33.889153 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6659bc-c15d-4fed-99bd-33ab9c25f553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:33.889846 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6659bc-c15d-4fed-99bd-33ab9c25f553 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:33.890166 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6659bc-c15d-4fed-99bd-33ab9c25f553 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:33.890438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6659bc-c15d-4fed-99bd-33ab9c25f553 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:33.898645 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6659bc-c15d-4fed-99bd-33ab9c25f553 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:33.900664 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6659bc-c15d-4fed-99bd-33ab9c25f553 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:33.910934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-0e6659bc-c15d-4fed-99bd-33ab9c25f553 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 11:49:33.910934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:33.910934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:33.910934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:33.910934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:33.910934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:33.910934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:33.910934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:33.910934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:33.910934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:33.910934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:33.910934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:33.910934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:49:33.910934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:49:33.910934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:49:33.910934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:49:33.912708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:33.912708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:33.912708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:49:33.912708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:49:33.912708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:49:33.912708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:49:33.912708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:33.912708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:33.912708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:33.912708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:33.912708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:33.912708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:33.912708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:33.912708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:33.912708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:33.912708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:33.912708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:33.912708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:33.913979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:33.913979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:33.913979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:49:33.913979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:49:33.913979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:49:33.913979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:49:33.913979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:49:33.913979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:49:33.913979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 11:49:33.913979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:49:33.913979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 608/1212] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 11:49:33.918182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6929bab-8c51-4ce4-9971-2fdd47470d5b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:34.020151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6929bab-8c51-4ce4-9971-2fdd47470d5b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:34.054915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8631f6d4-11f2-484c-8510-3c0c4300fcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:34.055567 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631f6d4-11f2-484c-8510-3c0c4300fcf6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:34.055848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631f6d4-11f2-484c-8510-3c0c4300fcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:34.056127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631f6d4-11f2-484c-8510-3c0c4300fcf6 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:34.056855 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8631f6d4-11f2-484c-8510-3c0c4300fcf6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:34.057164 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8631f6d4-11f2-484c-8510-3c0c4300fcf6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:34.058616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8631f6d4-11f2-484c-8510-3c0c4300fcf6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:34.075960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 605/1213] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Sep 14 11:49:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:34.082122 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6659bc-c15d-4fed-99bd-33ab9c25f553 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:34.182063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6659bc-c15d-4fed-99bd-33ab9c25f553 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:34.215218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1c336-f266-4c13-9455-69946eaaae44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:34.216347 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1c336-f266-4c13-9455-69946eaaae44 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:34.216630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1c336-f266-4c13-9455-69946eaaae44 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:34.216906 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1c336-f266-4c13-9455-69946eaaae44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:34.246720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:34.285035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1c336-f266-4c13-9455-69946eaaae44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:34.286489 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1c336-f266-4c13-9455-69946eaaae44 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:34.386812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 609/1214] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:49:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:49:34.392874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8631f6d4-11f2-484c-8510-3c0c4300fcf6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:34.411748 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:34.492904 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8631f6d4-11f2-484c-8510-3c0c4300fcf6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:34.533494 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde82b8-36f3-4597-b40f-437b8dfaf3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:34.535224 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde82b8-36f3-4597-b40f-437b8dfaf3b4 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:34.535224 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde82b8-36f3-4597-b40f-437b8dfaf3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:34.535224 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde82b8-36f3-4597-b40f-437b8dfaf3b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:34.545624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde82b8-36f3-4597-b40f-437b8dfaf3b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:34.547115 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde82b8-36f3-4597-b40f-437b8dfaf3b4 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:34.562092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-ffde82b8-36f3-4597-b40f-437b8dfaf3b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:34.562092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:34.562092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:34.562092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:34.562092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:34.562092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:34.562092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:34.562092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:34.562092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:34.562092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:34.562092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:34.562092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:34.562092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:49:34.562092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:49:34.562092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:49:34.562092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:49:34.567451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:34.567451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:34.567451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:34.567451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:34.567451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:34.567451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:34.567451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:34.567451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:34.567451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:34.567451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:34.567451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:34.567451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:34.567451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:34.567451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:34.567451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:34.567451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:34.567451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:49:34.568572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:49:34.568572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:34.568572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:49:34.568572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 606/1215] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:34.571188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1c336-f266-4c13-9455-69946eaaae44 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:34.639516 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:34.678610 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1c336-f266-4c13-9455-69946eaaae44 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:34.714666 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560443b4-8082-4f98-af86-00d6862a279a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:34.715273 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560443b4-8082-4f98-af86-00d6862a279a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:34.715585 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560443b4-8082-4f98-af86-00d6862a279a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:34.715864 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560443b4-8082-4f98-af86-00d6862a279a None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:34.716516 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560443b4-8082-4f98-af86-00d6862a279a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:34.716822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560443b4-8082-4f98-af86-00d6862a279a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:34.718229 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560443b4-8082-4f98-af86-00d6862a279a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:34.729940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 610/1216] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:34.735877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde82b8-36f3-4597-b40f-437b8dfaf3b4 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:34.809943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:34.844247 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde82b8-36f3-4597-b40f-437b8dfaf3b4 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:34.879170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5788316-d40c-4126-8dcc-0bf5b4f0757d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:34.879807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5788316-d40c-4126-8dcc-0bf5b4f0757d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:34.880090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5788316-d40c-4126-8dcc-0bf5b4f0757d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:34.880366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5788316-d40c-4126-8dcc-0bf5b4f0757d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:34.887200 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5788316-d40c-4126-8dcc-0bf5b4f0757d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:34.889195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5788316-d40c-4126-8dcc-0bf5b4f0757d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:34.900824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d5788316-d40c-4126-8dcc-0bf5b4f0757d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:49:34.900824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:34.900824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:34.900824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:34.900824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:34.900824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:34.900824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:34.900824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:34.900824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:34.900824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:34.900824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:34.900824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:34.900824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:49:34.900824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:49:34.900824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:49:34.906859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:49:34.906859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:34.906859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:34.906859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:34.906859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:34.906859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:34.906859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:34.906859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:34.906859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:34.906859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:34.906859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:34.906859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:34.906859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:34.906859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:34.906859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:34.906859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:34.906859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:34.907805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:49:34.907805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:49:34.907805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:49:34.907805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:49:34.907805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:49:34.907805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:49:34.907805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:49:34.907805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:49:34.907805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 607/1217] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:49:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:34.908373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560443b4-8082-4f98-af86-00d6862a279a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:35.020763 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560443b4-8082-4f98-af86-00d6862a279a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:35.044246 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:35.056337 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b8eb7-fdba-4caa-b27f-3fbb1395d883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:35.056942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b8eb7-fdba-4caa-b27f-3fbb1395d883 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:35.057221 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b8eb7-fdba-4caa-b27f-3fbb1395d883 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:35.057494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b8eb7-fdba-4caa-b27f-3fbb1395d883 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:35.058223 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b8eb7-fdba-4caa-b27f-3fbb1395d883 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:35.058555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b8eb7-fdba-4caa-b27f-3fbb1395d883 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:35.060850 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b8eb7-fdba-4caa-b27f-3fbb1395d883 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:35.073081 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 611/1218] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:49:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:35.079464 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5788316-d40c-4126-8dcc-0bf5b4f0757d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:35.182504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5788316-d40c-4126-8dcc-0bf5b4f0757d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:35.211605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:35.218183 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db0cdf-d730-41ed-b94c-67ba9b474ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:35.218794 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db0cdf-d730-41ed-b94c-67ba9b474ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:35.219077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db0cdf-d730-41ed-b94c-67ba9b474ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:35.219379 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db0cdf-d730-41ed-b94c-67ba9b474ab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:35.220108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5db0cdf-d730-41ed-b94c-67ba9b474ab8 None admin] RBAC: Adding query filter params (role.id=d1bb55f25dff4694b8675e0ef5fc4ec7, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=0ca10dd7ed714edb8b87017deecc41b6) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:35.220413 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5db0cdf-d730-41ed-b94c-67ba9b474ab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:35.221838 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5db0cdf-d730-41ed-b94c-67ba9b474ab8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:35.227722 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 608/1219] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:49:35 2022] GET /identity/v3/role_assignments?user.id=0ca10dd7ed714edb8b87017deecc41b6&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:37.396052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695c1a2-efb8-4370-86c4-0fe6596ae0ce None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:37.396610 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695c1a2-efb8-4370-86c4-0fe6596ae0ce None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:37.396610 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695c1a2-efb8-4370-86c4-0fe6596ae0ce None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:37.398020 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 612/1220] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:49:37.403407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8404c91-4aff-4cec-8587-3f4c4fd1a1cf None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:37.403837 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8404c91-4aff-4cec-8587-3f4c4fd1a1cf None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:37.404008 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8404c91-4aff-4cec-8587-3f4c4fd1a1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:37.423018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e8404c91-4aff-4cec-8587-3f4c4fd1a1cf None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:49:37.426600 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e8404c91-4aff-4cec-8587-3f4c4fd1a1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:49:37.492510 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 609/1221] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:49:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 11:49:37.762238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695c1a2-efb8-4370-86c4-0fe6596ae0ce None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:37.868175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695c1a2-efb8-4370-86c4-0fe6596ae0ce None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:37.903703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52985a4-cb48-4239-a456-26ea211d646c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:37.904380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52985a4-cb48-4239-a456-26ea211d646c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:37.904680 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52985a4-cb48-4239-a456-26ea211d646c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:37.904953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52985a4-cb48-4239-a456-26ea211d646c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:37.916936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52985a4-cb48-4239-a456-26ea211d646c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:49:37.923086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-c52985a4-cb48-4239-a456-26ea211d646c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:49:37.923086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:37.923086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:37.923086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:37.923086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:37.923086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:37.923086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:37.923086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:37.923086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:37.923086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:37.923086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:37.923086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:37.923086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:49:37.923086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:49:37.923086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:49:37.923086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:49:37.924791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:49:37.924791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:49:37.924791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:49:37.924791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:49:37.924791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:37.924791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:37.924791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:37.924791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:37.924791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:37.924791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:37.924791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:37.924791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:37.924791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:37.924791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:37.924791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:37.924791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:37.925737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:37.925737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:37.925737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:37.925737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:37.925737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:49:37.925737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:49:37.925737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:49:37.925737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:49:37.925737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:49:37.925737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:49:37.925737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:49:37.925737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:49:37.925737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 613/1222] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:49:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:37.930815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8404c91-4aff-4cec-8587-3f4c4fd1a1cf None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:38.035684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8404c91-4aff-4cec-8587-3f4c4fd1a1cf None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:38.061235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:38.071743 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0836807-22e6-4cf1-8d83-cb55ad4af1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:38.072418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0836807-22e6-4cf1-8d83-cb55ad4af1b6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:38.072705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0836807-22e6-4cf1-8d83-cb55ad4af1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:38.072982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0836807-22e6-4cf1-8d83-cb55ad4af1b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:38.073652 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0836807-22e6-4cf1-8d83-cb55ad4af1b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:38.073989 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0836807-22e6-4cf1-8d83-cb55ad4af1b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:38.075888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0836807-22e6-4cf1-8d83-cb55ad4af1b6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:38.086545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 610/1223] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:49:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:38.093136 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52985a4-cb48-4239-a456-26ea211d646c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:38.139750 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:38.197259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52985a4-cb48-4239-a456-26ea211d646c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:38.239202 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e96025-15f0-495a-842d-a4185167b42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:38.239840 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e96025-15f0-495a-842d-a4185167b42d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:38.240124 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e96025-15f0-495a-842d-a4185167b42d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:38.240405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e96025-15f0-495a-842d-a4185167b42d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:38.253135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e96025-15f0-495a-842d-a4185167b42d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:38.254633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e96025-15f0-495a-842d-a4185167b42d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:38.269284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-e7e96025-15f0-495a-842d-a4185167b42d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:38.269284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:38.269284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:38.269284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:38.269284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:38.269284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:38.269284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:38.269284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:38.269284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:38.269284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:38.269284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:38.269284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:38.269284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:49:38.269284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:49:38.269284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:49:38.269284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:49:38.271098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:38.271098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:38.271098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:38.271098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:38.271098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:38.271098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:38.271098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:38.271098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:38.271098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:38.271098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:38.271098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:38.271098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:38.271098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:38.271098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:38.271098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:38.271098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:38.271098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:49:38.272740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:49:38.272740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:38.272740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:49:38.272740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 614/1224] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:38.276928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0836807-22e6-4cf1-8d83-cb55ad4af1b6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:38.398696 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0836807-22e6-4cf1-8d83-cb55ad4af1b6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:38.433506 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec716f-19e0-41f4-bd22-9c33e214b441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:38.434726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec716f-19e0-41f4-bd22-9c33e214b441 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:38.434935 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec716f-19e0-41f4-bd22-9c33e214b441 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:38.435273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec716f-19e0-41f4-bd22-9c33e214b441 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:38.435917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec716f-19e0-41f4-bd22-9c33e214b441 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:38.436338 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec716f-19e0-41f4-bd22-9c33e214b441 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:38.437771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec716f-19e0-41f4-bd22-9c33e214b441 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:38.450674 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 611/1225] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:38.457361 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e96025-15f0-495a-842d-a4185167b42d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:38.553746 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:38.561114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e96025-15f0-495a-842d-a4185167b42d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:38.595107 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04193ac0-07d6-44ac-b391-84e1f7897e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:38.595749 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04193ac0-07d6-44ac-b391-84e1f7897e05 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:38.596028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04193ac0-07d6-44ac-b391-84e1f7897e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:38.596300 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04193ac0-07d6-44ac-b391-84e1f7897e05 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:38.604190 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04193ac0-07d6-44ac-b391-84e1f7897e05 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:38.605627 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04193ac0-07d6-44ac-b391-84e1f7897e05 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:38.616837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-04193ac0-07d6-44ac-b391-84e1f7897e05 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 11:49:38.616837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:38.616837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:38.616837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:38.616837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:38.616837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:38.616837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:38.616837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:38.616837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:38.616837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:38.616837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:38.616837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:38.616837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:49:38.616837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:49:38.616837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:49:38.616837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:49:38.622176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:38.622176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:38.622176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:49:38.622176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:49:38.622176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:49:38.622176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:49:38.622176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:38.622176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:38.622176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:38.622176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:38.622176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:38.622176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:38.622176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:38.622176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:38.622176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:38.622176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:38.622176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:38.622176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:38.624069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:38.624069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:38.624069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:49:38.624069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:49:38.624069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:49:38.624069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:49:38.624069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:49:38.624069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:49:38.624069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 11:49:38.624069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:49:38.624069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 615/1226] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:38.625299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec716f-19e0-41f4-bd22-9c33e214b441 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:38.673225 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:38.724570 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec716f-19e0-41f4-bd22-9c33e214b441 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:38.759348 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64590e9d-17a5-4273-9792-1bb805fd9061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:38.760315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64590e9d-17a5-4273-9792-1bb805fd9061 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:38.760758 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64590e9d-17a5-4273-9792-1bb805fd9061 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:38.761194 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64590e9d-17a5-4273-9792-1bb805fd9061 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:38.762500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64590e9d-17a5-4273-9792-1bb805fd9061 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:38.763006 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64590e9d-17a5-4273-9792-1bb805fd9061 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:38.764604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64590e9d-17a5-4273-9792-1bb805fd9061 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:38.785290 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 612/1227] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Sep 14 11:49:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:49:38.791363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04193ac0-07d6-44ac-b391-84e1f7897e05 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:38.896988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04193ac0-07d6-44ac-b391-84e1f7897e05 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:38.938863 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6ccf6-5dec-41bf-afde-0017b598e360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:38.939413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6ccf6-5dec-41bf-afde-0017b598e360 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:38.939689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6ccf6-5dec-41bf-afde-0017b598e360 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:38.939962 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6ccf6-5dec-41bf-afde-0017b598e360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:38.950635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6ccf6-5dec-41bf-afde-0017b598e360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:38.952066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6ccf6-5dec-41bf-afde-0017b598e360 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:38.966100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-8dd6ccf6-5dec-41bf-afde-0017b598e360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:38.966100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:38.966100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:38.966100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:38.966100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:38.966100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:38.966100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:38.966100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:38.966100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:38.966100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:38.966100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:38.966100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:38.966100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:49:38.966100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:49:38.966100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:49:38.966100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:49:38.967301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:38.967301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:38.967301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:38.967301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:38.967301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:38.967301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:38.967301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:38.967301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:38.967301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:38.967301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:38.967301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:38.967301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:38.967301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:38.967301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:38.967301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:38.967301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:38.967301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:49:38.968450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:49:38.968450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:38.968450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:49:38.968450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 616/1228] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:38.973064 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64590e9d-17a5-4273-9792-1bb805fd9061 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:39.076817 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64590e9d-17a5-4273-9792-1bb805fd9061 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:39.079176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:39.111647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1c3a6-d869-4b7d-86d0-e798cb9646fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:39.112309 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1c3a6-d869-4b7d-86d0-e798cb9646fe None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:39.112595 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1c3a6-d869-4b7d-86d0-e798cb9646fe None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:39.112959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1c3a6-d869-4b7d-86d0-e798cb9646fe None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:39.113669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1c3a6-d869-4b7d-86d0-e798cb9646fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:39.114134 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1c3a6-d869-4b7d-86d0-e798cb9646fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:39.124746 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1c3a6-d869-4b7d-86d0-e798cb9646fe None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:39.128561 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 613/1229] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:39.134585 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6ccf6-5dec-41bf-afde-0017b598e360 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:39.144371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:39.240526 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6ccf6-5dec-41bf-afde-0017b598e360 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:39.277030 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe0fd8-8499-4c31-a07f-f7caa58ddbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:39.277659 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe0fd8-8499-4c31-a07f-f7caa58ddbe0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:39.278516 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe0fd8-8499-4c31-a07f-f7caa58ddbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:39.278787 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe0fd8-8499-4c31-a07f-f7caa58ddbe0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:39.285480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe0fd8-8499-4c31-a07f-f7caa58ddbe0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:39.286949 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe0fd8-8499-4c31-a07f-f7caa58ddbe0 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:39.300057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-bbfe0fd8-8499-4c31-a07f-f7caa58ddbe0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:49:39.300057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:39.300057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:39.300057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:39.300057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:39.300057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:39.300057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:39.300057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:39.300057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:39.300057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:39.300057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:39.300057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:39.300057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:49:39.300057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:49:39.300057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:49:39.301193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:49:39.301193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:39.301193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:39.301193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:39.301193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:39.301193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:39.301193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:39.301193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:39.301193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:39.301193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:39.301193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:39.301193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:39.301193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:39.301193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:39.301193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:39.301193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:39.301193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:39.302309 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:49:39.302309 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:49:39.302309 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:49:39.302309 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:49:39.302309 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:49:39.302309 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:49:39.302309 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:49:39.302309 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:49:39.302309 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 617/1230] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:49:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:39.307201 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1c3a6-d869-4b7d-86d0-e798cb9646fe None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:39.424235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1c3a6-d869-4b7d-86d0-e798cb9646fe None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:39.461688 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed6268-0054-4c58-9521-0d0b2debf27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:39.462391 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed6268-0054-4c58-9521-0d0b2debf27a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:39.462583 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed6268-0054-4c58-9521-0d0b2debf27a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:39.462857 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed6268-0054-4c58-9521-0d0b2debf27a None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:39.463557 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed6268-0054-4c58-9521-0d0b2debf27a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:39.463865 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed6268-0054-4c58-9521-0d0b2debf27a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:39.465933 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed6268-0054-4c58-9521-0d0b2debf27a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:39.478923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 614/1231] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:49:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:49:39.485380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe0fd8-8499-4c31-a07f-f7caa58ddbe0 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:39.556926 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:39.590586 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe0fd8-8499-4c31-a07f-f7caa58ddbe0 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:39.624848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ea105-67ef-45fa-8290-6662dee30cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:39.625504 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ea105-67ef-45fa-8290-6662dee30cb8 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:39.625786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ea105-67ef-45fa-8290-6662dee30cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:39.626086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ea105-67ef-45fa-8290-6662dee30cb8 None admin] PATH_INFO: `/v3/projects/449a5fac1acd478abea544446c08ecd9/users/0ca10dd7ed714edb8b87017deecc41b6/roles/d1bb55f25dff4694b8675e0ef5fc4ec7` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:39.641719 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ea105-67ef-45fa-8290-6662dee30cb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=449a5fac1acd478abea544446c08ecd9, user_id=0ca10dd7ed714edb8b87017deecc41b6, role_id=d1bb55f25dff4694b8675e0ef5fc4ec7)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:39.643197 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ea105-67ef-45fa-8290-6662dee30cb8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:39.708530 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 618/1232] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:49:39 2022] PUT /identity/v3/projects/449a5fac1acd478abea544446c08ecd9/users/0ca10dd7ed714edb8b87017deecc41b6/roles/d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:49:41.915022 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568e167-b429-43e3-b911-8a109d8aaa60 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:41.915554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568e167-b429-43e3-b911-8a109d8aaa60 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:41.915554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568e167-b429-43e3-b911-8a109d8aaa60 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:41.916914 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 615/1233] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:49:41.923296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ec971-0c4d-4c7d-8b23-50cb342ef371 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:41.923743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ec971-0c4d-4c7d-8b23-50cb342ef371 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:41.923856 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ec971-0c4d-4c7d-8b23-50cb342ef371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:41.943110 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e80ec971-0c4d-4c7d-8b23-50cb342ef371 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:49:41.944658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e80ec971-0c4d-4c7d-8b23-50cb342ef371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:49:42.015434 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 619/1234] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:49:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:49:42.288618 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568e167-b429-43e3-b911-8a109d8aaa60 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:42.314647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:42.393829 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568e167-b429-43e3-b911-8a109d8aaa60 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:42.427576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75ce5a-6151-49c1-b82e-33029a6c2c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:42.428198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75ce5a-6151-49c1-b82e-33029a6c2c4b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:42.428475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75ce5a-6151-49c1-b82e-33029a6c2c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:42.428745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75ce5a-6151-49c1-b82e-33029a6c2c4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:42.439256 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c75ce5a-6151-49c1-b82e-33029a6c2c4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:49:42.444036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-0c75ce5a-6151-49c1-b82e-33029a6c2c4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:49:42.444036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:42.444036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:42.444036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:42.444036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:42.444036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:42.444036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:42.444036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:42.444036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:42.444036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:42.444036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:42.444036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:42.444036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:49:42.444036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:49:42.444036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:49:42.444036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:49:42.445559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:49:42.445559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:49:42.445559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:49:42.445559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:49:42.445559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:42.445559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:42.445559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:42.445559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:42.445559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:42.445559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:42.445559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:42.445559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:42.445559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:42.445559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:42.445559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:42.445559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:42.446505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:42.446505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:42.446505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:42.446505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:42.446505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:49:42.446505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:49:42.446505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:49:42.446505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:49:42.446505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:49:42.446505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:49:42.446505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 11:49:42.446505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:49:42.446505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 616/1235] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 14 11:49:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:42.451903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ec971-0c4d-4c7d-8b23-50cb342ef371 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:42.499501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:42.554936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ec971-0c4d-4c7d-8b23-50cb342ef371 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:42.596215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7a942-f7b7-4d6c-a900-f9f75a4a7da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:42.596837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7a942-f7b7-4d6c-a900-f9f75a4a7da7 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:42.597124 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7a942-f7b7-4d6c-a900-f9f75a4a7da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:42.597403 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7a942-f7b7-4d6c-a900-f9f75a4a7da7 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:42.598111 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7a942-f7b7-4d6c-a900-f9f75a4a7da7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:42.598423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7a942-f7b7-4d6c-a900-f9f75a4a7da7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:42.599796 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7a942-f7b7-4d6c-a900-f9f75a4a7da7 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:42.608298 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 620/1236] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 14 11:49:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:42.614219 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75ce5a-6151-49c1-b82e-33029a6c2c4b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:42.712957 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:42.731264 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75ce5a-6151-49c1-b82e-33029a6c2c4b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:42.765706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c415f2c-40a1-4522-81f2-61ce95483686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:42.766374 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c415f2c-40a1-4522-81f2-61ce95483686 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:42.766655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c415f2c-40a1-4522-81f2-61ce95483686 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:42.766926 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c415f2c-40a1-4522-81f2-61ce95483686 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:42.777227 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c415f2c-40a1-4522-81f2-61ce95483686 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:42.779247 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c415f2c-40a1-4522-81f2-61ce95483686 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:42.794508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-4c415f2c-40a1-4522-81f2-61ce95483686 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:42.794508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:42.794508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:42.794508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:42.794508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:42.794508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:42.794508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:42.794508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:42.794508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:42.794508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:42.794508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:42.794508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:42.794508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:49:42.794508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:49:42.794508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:49:42.794508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:49:42.796018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:42.796018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:42.796018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:42.796018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:42.796018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:42.796018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:42.796018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:42.796018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:42.796018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:42.796018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:42.796018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:42.796018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:42.796018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:42.796018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:42.796018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:42.796018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:42.796018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:49:42.797153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:49:42.797153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:42.797153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:49:42.797153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 617/1237] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:42.801753 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7a942-f7b7-4d6c-a900-f9f75a4a7da7 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:42.909875 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7a942-f7b7-4d6c-a900-f9f75a4a7da7 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:42.919167 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:42.945923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0efd0-bfcf-4c1f-bcda-e112f4035509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:42.946609 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0efd0-bfcf-4c1f-bcda-e112f4035509 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:42.946816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0efd0-bfcf-4c1f-bcda-e112f4035509 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:42.947096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0efd0-bfcf-4c1f-bcda-e112f4035509 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:42.947753 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0efd0-bfcf-4c1f-bcda-e112f4035509 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:42.948064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0efd0-bfcf-4c1f-bcda-e112f4035509 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:42.949450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0efd0-bfcf-4c1f-bcda-e112f4035509 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:42.962433 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 621/1238] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:42.968680 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c415f2c-40a1-4522-81f2-61ce95483686 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:43.074169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c415f2c-40a1-4522-81f2-61ce95483686 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:43.109359 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a4de4-6171-4cb5-affb-a0a1f39284a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:43.110017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a4de4-6171-4cb5-affb-a0a1f39284a3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:43.110298 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a4de4-6171-4cb5-affb-a0a1f39284a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:43.110569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a4de4-6171-4cb5-affb-a0a1f39284a3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:43.113369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:43.118842 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a4de4-6171-4cb5-affb-a0a1f39284a3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:43.120284 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a4de4-6171-4cb5-affb-a0a1f39284a3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:43.132328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-f40a4de4-6171-4cb5-affb-a0a1f39284a3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 11:49:43.132328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:43.132328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:43.132328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:43.132328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:43.132328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:43.132328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:43.132328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:43.132328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:43.132328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:43.132328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:43.132328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:43.132328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:49:43.132328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:49:43.132328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:49:43.132328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:49:43.134182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:43.134182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:43.134182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:49:43.134182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:49:43.134182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:49:43.134182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:49:43.134182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:43.134182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:43.134182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:43.134182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:43.134182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:43.134182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:43.134182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:43.134182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:43.134182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:43.134182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:43.134182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:43.134182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:43.135683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:43.135683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:43.135683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:49:43.135683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:49:43.135683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:49:43.135683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:49:43.135683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:49:43.135683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:49:43.135683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 11:49:43.135683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:49:43.135683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 618/1239] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:43.140034 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0efd0-bfcf-4c1f-bcda-e112f4035509 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:43.245304 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0efd0-bfcf-4c1f-bcda-e112f4035509 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:43.280994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea41d9e-051a-4f4f-a0d7-552d801fb616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:43.281598 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea41d9e-051a-4f4f-a0d7-552d801fb616 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:43.281901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea41d9e-051a-4f4f-a0d7-552d801fb616 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:43.282177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea41d9e-051a-4f4f-a0d7-552d801fb616 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:43.282965 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea41d9e-051a-4f4f-a0d7-552d801fb616 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:43.283269 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea41d9e-051a-4f4f-a0d7-552d801fb616 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:43.284702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea41d9e-051a-4f4f-a0d7-552d801fb616 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:43.303331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 622/1240] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Sep 14 11:49:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:43.309425 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a4de4-6171-4cb5-affb-a0a1f39284a3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:43.420759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a4de4-6171-4cb5-affb-a0a1f39284a3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:43.454843 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028b496-c14f-4884-815e-fbae60f7b1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:43.455597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028b496-c14f-4884-815e-fbae60f7b1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:43.455738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028b496-c14f-4884-815e-fbae60f7b1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:43.455993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028b496-c14f-4884-815e-fbae60f7b1f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:43.526541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:43.533973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028b496-c14f-4884-815e-fbae60f7b1f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:43.535478 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028b496-c14f-4884-815e-fbae60f7b1f2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:43.639376 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 619/1241] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:49:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:49:43.647995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea41d9e-051a-4f4f-a0d7-552d801fb616 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:43.686736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:43.764968 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea41d9e-051a-4f4f-a0d7-552d801fb616 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:43.800472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e221bc0-2e71-42f6-a667-9bef62138e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:43.801342 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e221bc0-2e71-42f6-a667-9bef62138e26 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:43.801623 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e221bc0-2e71-42f6-a667-9bef62138e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:43.801925 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e221bc0-2e71-42f6-a667-9bef62138e26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:43.812701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e221bc0-2e71-42f6-a667-9bef62138e26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:43.814174 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e221bc0-2e71-42f6-a667-9bef62138e26 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:43.831059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-8e221bc0-2e71-42f6-a667-9bef62138e26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:43.831059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:43.831059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:43.831059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:43.831059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:43.831059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:43.831059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:43.831059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:43.831059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:43.831059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:43.831059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:43.831059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:43.831059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:49:43.831059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:49:43.831059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:49:43.831059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:49:43.832257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:43.832257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:43.832257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:43.832257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:43.832257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:43.832257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:43.832257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:43.832257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:43.832257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:43.832257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:43.832257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:43.832257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:43.832257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:43.832257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:43.832257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:43.832257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:43.832257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:49:43.833330 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:49:43.833330 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:43.833330 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:49:43.833330 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 623/1242] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:43.840351 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028b496-c14f-4884-815e-fbae60f7b1f2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:43.940709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:43.946780 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028b496-c14f-4884-815e-fbae60f7b1f2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:43.981996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f1800-e2d2-4a93-832f-92ecf3827570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:43.982697 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f1800-e2d2-4a93-832f-92ecf3827570 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:43.982878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f1800-e2d2-4a93-832f-92ecf3827570 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:43.983157 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f1800-e2d2-4a93-832f-92ecf3827570 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:43.983802 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f1800-e2d2-4a93-832f-92ecf3827570 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:43.984108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f1800-e2d2-4a93-832f-92ecf3827570 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:43.985468 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f1800-e2d2-4a93-832f-92ecf3827570 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:43.997554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 620/1243] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:49:44.003792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e221bc0-2e71-42f6-a667-9bef62138e26 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:44.087651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:44.109892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e221bc0-2e71-42f6-a667-9bef62138e26 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:44.145726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5047974-f91b-430c-ad4f-21c6d5038e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:44.146396 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5047974-f91b-430c-ad4f-21c6d5038e83 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:44.146683 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5047974-f91b-430c-ad4f-21c6d5038e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:44.146956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5047974-f91b-430c-ad4f-21c6d5038e83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:44.153947 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5047974-f91b-430c-ad4f-21c6d5038e83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:44.155443 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5047974-f91b-430c-ad4f-21c6d5038e83 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:44.168876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-c5047974-f91b-430c-ad4f-21c6d5038e83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:49:44.168876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:44.168876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:44.168876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:44.168876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:44.168876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:44.168876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:44.168876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:44.168876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:44.168876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:44.168876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:44.168876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:44.168876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:49:44.168876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:49:44.168876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:49:44.171028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:49:44.171028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:44.171028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:44.171028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:44.171028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:44.171028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:44.171028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:44.171028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:44.171028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:44.171028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:44.171028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:44.171028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:44.171028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:44.171028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:44.171028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:44.171028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:44.171028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:44.173228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:49:44.173228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:49:44.173228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:49:44.173228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:49:44.173228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:49:44.173228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:49:44.173228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:49:44.173228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:49:44.173228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 624/1244] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:49:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:44.177833 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f1800-e2d2-4a93-832f-92ecf3827570 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:44.286461 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f1800-e2d2-4a93-832f-92ecf3827570 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:44.329319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb09c0-dce6-4fc5-9189-334d1e6c4969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:44.329963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb09c0-dce6-4fc5-9189-334d1e6c4969 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:44.330244 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb09c0-dce6-4fc5-9189-334d1e6c4969 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:44.330515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb09c0-dce6-4fc5-9189-334d1e6c4969 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:44.331204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfb09c0-dce6-4fc5-9189-334d1e6c4969 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:44.331509 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfb09c0-dce6-4fc5-9189-334d1e6c4969 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:44.332920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfb09c0-dce6-4fc5-9189-334d1e6c4969 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:44.346355 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 621/1245] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:49:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:44.352531 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5047974-f91b-430c-ad4f-21c6d5038e83 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:44.460264 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5047974-f91b-430c-ad4f-21c6d5038e83 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:44.502146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae68f5-760a-43ab-9c93-fde78382ea30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:44.502834 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae68f5-760a-43ab-9c93-fde78382ea30 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:44.503122 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae68f5-760a-43ab-9c93-fde78382ea30 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:44.503400 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae68f5-760a-43ab-9c93-fde78382ea30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:44.504148 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae68f5-760a-43ab-9c93-fde78382ea30 None admin] RBAC: Adding query filter params (role.id=d1bb55f25dff4694b8675e0ef5fc4ec7, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=0ca10dd7ed714edb8b87017deecc41b6) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:44.504461 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae68f5-760a-43ab-9c93-fde78382ea30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:44.505927 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae68f5-760a-43ab-9c93-fde78382ea30 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:44.514002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 625/1246] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:49:44 2022] GET /identity/v3/role_assignments?user.id=0ca10dd7ed714edb8b87017deecc41b6&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=d1bb55f25dff4694b8675e0ef5fc4ec7 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:46.800207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf9aef-fbb1-4def-bbd7-19a05debb71d None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:46.800747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf9aef-fbb1-4def-bbd7-19a05debb71d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:46.800747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf9aef-fbb1-4def-bbd7-19a05debb71d None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:46.802135 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 622/1247] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:49:46.807787 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d8e0c-013e-4cf3-8f27-e004ad5389bc None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:46.808273 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d8e0c-013e-4cf3-8f27-e004ad5389bc None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:46.808377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d8e0c-013e-4cf3-8f27-e004ad5389bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:46.817943 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:46.837365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-505d8e0c-013e-4cf3-8f27-e004ad5389bc None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:49:46.839121 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-505d8e0c-013e-4cf3-8f27-e004ad5389bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:49:46.920083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 626/1248] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:49:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:49:47.195414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf9aef-fbb1-4def-bbd7-19a05debb71d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:47.214446 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:47.312561 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf9aef-fbb1-4def-bbd7-19a05debb71d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:47.350521 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207cb42-1083-4304-aeec-705b71ce7840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:47.351145 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207cb42-1083-4304-aeec-705b71ce7840 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:47.351424 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207cb42-1083-4304-aeec-705b71ce7840 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:47.351697 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207cb42-1083-4304-aeec-705b71ce7840 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:47.363756 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f207cb42-1083-4304-aeec-705b71ce7840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:49:47.368537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-f207cb42-1083-4304-aeec-705b71ce7840 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:49:47.368537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:47.368537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:47.368537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:47.368537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:47.368537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:47.368537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:47.368537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:47.368537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:47.368537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:47.368537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:47.368537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:47.368537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:49:47.368537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:49:47.368537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:49:47.368537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:49:47.369651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:49:47.369651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:49:47.369651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:49:47.369651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:49:47.369651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:47.369651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:47.369651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:47.369651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:47.369651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:47.369651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:47.369651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:47.369651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:47.369651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:47.369651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:47.369651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:47.369651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:47.370591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:47.370591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:47.370591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:47.370591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:47.370591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:49:47.370591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:49:47.370591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:49:47.370591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:49:47.370591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:49:47.370591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:49:47.370591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:49:47.370591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:49:47.370591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 623/1249] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:49:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:47.376511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d8e0c-013e-4cf3-8f27-e004ad5389bc None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:47.481422 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d8e0c-013e-4cf3-8f27-e004ad5389bc None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:47.511243 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:47.516305 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fcfa77-70a7-43aa-a9bf-ec5d57933439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:47.516919 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fcfa77-70a7-43aa-a9bf-ec5d57933439 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:47.517240 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fcfa77-70a7-43aa-a9bf-ec5d57933439 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:47.517522 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fcfa77-70a7-43aa-a9bf-ec5d57933439 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:47.518228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fcfa77-70a7-43aa-a9bf-ec5d57933439 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:47.518535 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fcfa77-70a7-43aa-a9bf-ec5d57933439 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:47.519941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fcfa77-70a7-43aa-a9bf-ec5d57933439 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:47.528937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 627/1250] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:49:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:47.535158 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207cb42-1083-4304-aeec-705b71ce7840 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:47.617048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:47.646756 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207cb42-1083-4304-aeec-705b71ce7840 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:47.683392 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa76ad5-4c9a-49b5-be2b-f64c3aee782a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:47.684017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa76ad5-4c9a-49b5-be2b-f64c3aee782a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:47.684302 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa76ad5-4c9a-49b5-be2b-f64c3aee782a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:47.684573 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa76ad5-4c9a-49b5-be2b-f64c3aee782a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:47.694701 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa76ad5-4c9a-49b5-be2b-f64c3aee782a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:47.696167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa76ad5-4c9a-49b5-be2b-f64c3aee782a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:47.711320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-5aa76ad5-4c9a-49b5-be2b-f64c3aee782a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:47.711320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:47.711320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:47.711320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:47.711320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:47.711320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:47.711320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:47.711320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:47.711320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:47.711320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:47.711320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:47.711320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:47.711320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:49:47.711320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:49:47.711320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:49:47.711320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:49:47.712574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:47.712574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:47.712574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:47.712574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:47.712574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:47.712574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:47.712574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:47.712574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:47.712574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:47.712574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:47.712574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:47.712574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:47.712574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:47.712574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:47.712574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:47.712574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:47.712574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:49:47.713690 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:49:47.713690 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:47.713690 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:49:47.713690 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 624/1251] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:47.718993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fcfa77-70a7-43aa-a9bf-ec5d57933439 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:47.826941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fcfa77-70a7-43aa-a9bf-ec5d57933439 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:47.863098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89a284-4055-474e-80ca-7644822f80bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:47.863713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89a284-4055-474e-80ca-7644822f80bb None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:47.863998 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89a284-4055-474e-80ca-7644822f80bb None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:47.864275 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89a284-4055-474e-80ca-7644822f80bb None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:47.864926 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89a284-4055-474e-80ca-7644822f80bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:47.865231 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89a284-4055-474e-80ca-7644822f80bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:47.866641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89a284-4055-474e-80ca-7644822f80bb None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:47.878759 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 628/1252] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:47.884687 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa76ad5-4c9a-49b5-be2b-f64c3aee782a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:47.989059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa76ad5-4c9a-49b5-be2b-f64c3aee782a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:48.018648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:48.027502 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb86445-202b-486b-8ef0-77ee6a41b83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:48.028134 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb86445-202b-486b-8ef0-77ee6a41b83f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:48.028412 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb86445-202b-486b-8ef0-77ee6a41b83f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:48.028683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb86445-202b-486b-8ef0-77ee6a41b83f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:48.036669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb86445-202b-486b-8ef0-77ee6a41b83f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:48.038126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb86445-202b-486b-8ef0-77ee6a41b83f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:48.050180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-2eb86445-202b-486b-8ef0-77ee6a41b83f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 11:49:48.050180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:48.050180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:48.050180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:48.050180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:48.050180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:48.050180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:48.050180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:48.050180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:48.050180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:48.050180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:48.050180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:48.050180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:49:48.050180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:49:48.050180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:49:48.050180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:49:48.051213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:48.051213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:48.051213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:49:48.051213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:49:48.051213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:49:48.051213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:49:48.051213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:48.051213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:48.051213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:48.051213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:48.051213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:48.051213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:48.051213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:48.051213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:48.051213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:48.051213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:48.051213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:48.051213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:48.052261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:48.052261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:48.052261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:49:48.052261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:49:48.052261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:49:48.052261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:49:48.052261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:49:48.052261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:49:48.052261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 11:49:48.052261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:49:48.052261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 625/1253] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:48.057440 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89a284-4055-474e-80ca-7644822f80bb None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:48.113947 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:48.160492 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89a284-4055-474e-80ca-7644822f80bb None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:48.198731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac49197-4a7f-4df4-a208-e3f412b48ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:48.199345 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac49197-4a7f-4df4-a208-e3f412b48ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:48.199630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac49197-4a7f-4df4-a208-e3f412b48ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:48.199914 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac49197-4a7f-4df4-a208-e3f412b48ca6 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:48.200657 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac49197-4a7f-4df4-a208-e3f412b48ca6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:48.200963 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac49197-4a7f-4df4-a208-e3f412b48ca6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:48.202456 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac49197-4a7f-4df4-a208-e3f412b48ca6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:48.222048 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 629/1254] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Sep 14 11:49:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:48.228425 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb86445-202b-486b-8ef0-77ee6a41b83f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:48.332751 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb86445-202b-486b-8ef0-77ee6a41b83f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:48.369025 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318a5d5-eb01-4543-ac85-2d52fc3d22b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:48.369559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318a5d5-eb01-4543-ac85-2d52fc3d22b5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:48.369842 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318a5d5-eb01-4543-ac85-2d52fc3d22b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:48.370148 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318a5d5-eb01-4543-ac85-2d52fc3d22b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:48.428848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:48.445602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318a5d5-eb01-4543-ac85-2d52fc3d22b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:48.448026 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318a5d5-eb01-4543-ac85-2d52fc3d22b5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:48.562056 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 626/1255] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:49:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:49:48.568011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac49197-4a7f-4df4-a208-e3f412b48ca6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:48.672493 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac49197-4a7f-4df4-a208-e3f412b48ca6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:48.686830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:48.709839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec9d11-5bb2-40ac-b3df-7eaa9a142c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:48.710512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec9d11-5bb2-40ac-b3df-7eaa9a142c58 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:48.710796 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec9d11-5bb2-40ac-b3df-7eaa9a142c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:48.711074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec9d11-5bb2-40ac-b3df-7eaa9a142c58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:48.724294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec9d11-5bb2-40ac-b3df-7eaa9a142c58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:48.725766 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec9d11-5bb2-40ac-b3df-7eaa9a142c58 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:48.743776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-17ec9d11-5bb2-40ac-b3df-7eaa9a142c58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:48.743776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:48.743776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:48.743776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:48.743776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:48.743776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:48.743776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:48.743776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:48.743776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:48.743776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:48.743776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:48.743776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:48.743776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:49:48.743776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:49:48.743776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:49:48.743776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:49:48.745073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:48.745073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:48.745073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:48.745073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:48.745073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:48.745073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:48.745073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:48.745073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:48.745073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:48.745073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:48.745073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:48.745073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:48.745073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:48.745073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:48.745073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:48.745073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:48.745073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:49:48.746233 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:49:48.746233 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:48.746233 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:49:48.746233 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 630/1256] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:48.751153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318a5d5-eb01-4543-ac85-2d52fc3d22b5 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:48.849701 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:48.862191 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318a5d5-eb01-4543-ac85-2d52fc3d22b5 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:48.897431 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edb82a-ce5c-4e5f-a883-1f94234bf88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:48.898130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edb82a-ce5c-4e5f-a883-1f94234bf88a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:48.898347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edb82a-ce5c-4e5f-a883-1f94234bf88a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:48.898761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edb82a-ce5c-4e5f-a883-1f94234bf88a None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:48.899466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edb82a-ce5c-4e5f-a883-1f94234bf88a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:48.899779 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edb82a-ce5c-4e5f-a883-1f94234bf88a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:48.901264 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edb82a-ce5c-4e5f-a883-1f94234bf88a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:48.913625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 627/1257] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:48.919456 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec9d11-5bb2-40ac-b3df-7eaa9a142c58 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:49.028615 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec9d11-5bb2-40ac-b3df-7eaa9a142c58 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:49.064556 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c06caf-0c0f-435e-b905-1e39af83e086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:49.065191 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c06caf-0c0f-435e-b905-1e39af83e086 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:49.065471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c06caf-0c0f-435e-b905-1e39af83e086 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:49.065742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c06caf-0c0f-435e-b905-1e39af83e086 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:49.072997 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c06caf-0c0f-435e-b905-1e39af83e086 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:49.078456 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c06caf-0c0f-435e-b905-1e39af83e086 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:49.089924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-93c06caf-0c0f-435e-b905-1e39af83e086 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:49:49.089924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:49.089924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:49.089924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:49.089924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:49.089924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:49.089924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:49.089924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:49.089924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:49.089924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:49.089924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:49.089924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:49.089924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:49:49.089924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:49:49.089924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:49:49.091795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:49:49.091795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:49.091795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:49.091795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:49.091795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:49.091795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:49.091795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:49.091795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:49.091795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:49.091795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:49.091795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:49.091795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:49.091795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:49.091795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:49.091795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:49.091795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:49.091795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:49.092986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:49:49.092986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:49:49.092986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:49:49.092986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:49:49.092986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:49:49.092986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:49:49.092986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:49:49.092986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:49:49.092986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 631/1258] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:49:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:49.097663 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edb82a-ce5c-4e5f-a883-1f94234bf88a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:49.205228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edb82a-ce5c-4e5f-a883-1f94234bf88a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:49.243711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90db418-84a7-4004-bc80-98953b8c3f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:49.244328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90db418-84a7-4004-bc80-98953b8c3f56 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:49.244611 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90db418-84a7-4004-bc80-98953b8c3f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:49.244887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90db418-84a7-4004-bc80-98953b8c3f56 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:49.245585 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90db418-84a7-4004-bc80-98953b8c3f56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:49.245915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90db418-84a7-4004-bc80-98953b8c3f56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:49.247367 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90db418-84a7-4004-bc80-98953b8c3f56 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:49.259803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 628/1259] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:49:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:49:49.266064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c06caf-0c0f-435e-b905-1e39af83e086 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:49.293348 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:49.378547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c06caf-0c0f-435e-b905-1e39af83e086 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:49.412493 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509fa1d-b739-4401-a733-f68800c51deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:49.413090 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509fa1d-b739-4401-a733-f68800c51deb None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:49.413373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509fa1d-b739-4401-a733-f68800c51deb None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:49.413648 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509fa1d-b739-4401-a733-f68800c51deb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:49.414400 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509fa1d-b739-4401-a733-f68800c51deb None admin] RBAC: Adding query filter params (role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=0ca10dd7ed714edb8b87017deecc41b6) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:49.414708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509fa1d-b739-4401-a733-f68800c51deb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:49.416228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509fa1d-b739-4401-a733-f68800c51deb None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:49.422744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 632/1260] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:49:49 2022] GET /identity/v3/role_assignments?user.id=0ca10dd7ed714edb8b87017deecc41b6&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:51.571520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94efc964-d5dd-4c66-b908-801da04d2b6d None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:51.572089 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94efc964-d5dd-4c66-b908-801da04d2b6d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:51.572089 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94efc964-d5dd-4c66-b908-801da04d2b6d None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:51.573402 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 629/1261] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:49:51.579380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7253dc-27a2-41cf-8777-576c12fe8ea7 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:51.579868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7253dc-27a2-41cf-8777-576c12fe8ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:51.580007 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7253dc-27a2-41cf-8777-576c12fe8ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:51.599211 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-6e7253dc-27a2-41cf-8777-576c12fe8ea7 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:49:51.600470 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6e7253dc-27a2-41cf-8777-576c12fe8ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:49:51.670514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 633/1262] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:49:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:49:51.934902 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94efc964-d5dd-4c66-b908-801da04d2b6d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:51.945962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:52.047620 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94efc964-d5dd-4c66-b908-801da04d2b6d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:52.082584 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb69604-5358-41b7-b04e-25ba06fc5e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:52.083133 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb69604-5358-41b7-b04e-25ba06fc5e21 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:52.083419 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb69604-5358-41b7-b04e-25ba06fc5e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:52.083701 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb69604-5358-41b7-b04e-25ba06fc5e21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:52.094802 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb69604-5358-41b7-b04e-25ba06fc5e21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:49:52.099732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-4fb69604-5358-41b7-b04e-25ba06fc5e21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:49:52.099732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:52.099732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:52.099732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:52.099732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:52.099732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:52.099732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:52.099732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:52.099732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:52.099732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:52.099732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:52.099732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:52.099732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:49:52.099732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:49:52.099732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:49:52.099732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:49:52.100793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:49:52.100793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:49:52.100793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:49:52.100793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:49:52.100793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:52.100793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:52.100793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:52.100793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:52.100793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:52.100793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:52.100793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:52.100793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:52.100793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:52.100793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:52.100793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:52.100793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:52.101836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:52.101836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:52.101836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:52.101836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:52.101836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:49:52.101836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:49:52.101836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:49:52.101836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:49:52.101836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:49:52.101836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:49:52.101836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:49:52.101836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:49:52.101836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 630/1263] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:49:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:52.107217 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7253dc-27a2-41cf-8777-576c12fe8ea7 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:52.128160 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:52.212880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7253dc-27a2-41cf-8777-576c12fe8ea7 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:52.250894 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f94d4-0a20-4d0b-868a-839d8b15f11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:52.251510 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f94d4-0a20-4d0b-868a-839d8b15f11b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:52.251822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f94d4-0a20-4d0b-868a-839d8b15f11b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:52.252103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f94d4-0a20-4d0b-868a-839d8b15f11b None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:52.252782 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f94d4-0a20-4d0b-868a-839d8b15f11b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:52.253091 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f94d4-0a20-4d0b-868a-839d8b15f11b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:52.254608 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f94d4-0a20-4d0b-868a-839d8b15f11b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:52.264327 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 634/1264] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:49:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:52.270459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb69604-5358-41b7-b04e-25ba06fc5e21 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:52.347846 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:52.383468 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb69604-5358-41b7-b04e-25ba06fc5e21 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:52.419918 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bd534-7172-4eb3-b597-3268c78a575f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:52.420524 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bd534-7172-4eb3-b597-3268c78a575f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:52.420805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bd534-7172-4eb3-b597-3268c78a575f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:52.421070 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bd534-7172-4eb3-b597-3268c78a575f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:52.432014 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721bd534-7172-4eb3-b597-3268c78a575f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:52.433455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721bd534-7172-4eb3-b597-3268c78a575f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:52.449714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-721bd534-7172-4eb3-b597-3268c78a575f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:52.449714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:52.449714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:52.449714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:52.449714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:52.449714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:52.449714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:52.449714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:52.449714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:52.449714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:52.449714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:52.449714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:52.449714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:49:52.449714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:49:52.449714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:49:52.449714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:49:52.450862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:52.450862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:52.450862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:52.450862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:52.450862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:52.450862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:52.450862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:52.450862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:52.450862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:52.450862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:52.450862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:52.450862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:52.450862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:52.450862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:52.450862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:52.450862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:52.450862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:49:52.452077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:49:52.452077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:52.452077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:49:52.452077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 631/1265] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:52.456881 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f94d4-0a20-4d0b-868a-839d8b15f11b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:52.542834 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:52.571224 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f94d4-0a20-4d0b-868a-839d8b15f11b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:52.606333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed174227-024b-458a-87b9-1807e674d618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:52.607180 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed174227-024b-458a-87b9-1807e674d618 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:52.607180 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed174227-024b-458a-87b9-1807e674d618 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:52.607556 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed174227-024b-458a-87b9-1807e674d618 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:52.608459 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed174227-024b-458a-87b9-1807e674d618 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:52.608549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed174227-024b-458a-87b9-1807e674d618 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:52.609995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed174227-024b-458a-87b9-1807e674d618 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:52.626097 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 635/1266] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:52.631625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bd534-7172-4eb3-b597-3268c78a575f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:52.745826 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bd534-7172-4eb3-b597-3268c78a575f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:52.770746 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:52.785427 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6f527-d5e8-4d07-94b6-31d7d739c89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:52.786087 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6f527-d5e8-4d07-94b6-31d7d739c89c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:52.786372 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6f527-d5e8-4d07-94b6-31d7d739c89c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:52.786655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6f527-d5e8-4d07-94b6-31d7d739c89c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:52.795480 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6f527-d5e8-4d07-94b6-31d7d739c89c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:52.796951 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6f527-d5e8-4d07-94b6-31d7d739c89c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:52.809112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d0a6f527-d5e8-4d07-94b6-31d7d739c89c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 11:49:52.809112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:52.809112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:52.809112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:52.809112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:52.809112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:52.809112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:52.809112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:52.809112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:52.809112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:52.809112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:52.809112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:52.809112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:49:52.809112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:49:52.809112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:49:52.809112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:49:52.810401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:52.810401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:52.810401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:49:52.810401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:49:52.810401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:49:52.810401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:49:52.810401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:52.810401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:52.810401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:52.810401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:52.810401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:52.810401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:52.810401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:52.810401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:52.810401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:52.810401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:52.810401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:52.810401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:52.811624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:52.811624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:52.811624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:49:52.811624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:49:52.811624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:49:52.811624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:49:52.811624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:49:52.811624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:49:52.811624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 11:49:52.811624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:49:52.811624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 632/1267] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:52.818962 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed174227-024b-458a-87b9-1807e674d618 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:52.927338 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed174227-024b-458a-87b9-1807e674d618 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:52.967488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1967bc-a7b8-4259-ad4c-9d754a56e7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:52.968096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1967bc-a7b8-4259-ad4c-9d754a56e7ae None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:52.968383 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1967bc-a7b8-4259-ad4c-9d754a56e7ae None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:52.968649 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1967bc-a7b8-4259-ad4c-9d754a56e7ae None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:52.969386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1967bc-a7b8-4259-ad4c-9d754a56e7ae None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:52.969686 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1967bc-a7b8-4259-ad4c-9d754a56e7ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:52.971392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1967bc-a7b8-4259-ad4c-9d754a56e7ae None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:52.980723 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:52.990381 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 636/1268] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Sep 14 11:49:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:52.996544 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6f527-d5e8-4d07-94b6-31d7d739c89c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:53.102727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6f527-d5e8-4d07-94b6-31d7d739c89c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:53.136657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98eed5-a2f3-443b-bbfe-17a7123fc763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:53.137278 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98eed5-a2f3-443b-bbfe-17a7123fc763 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:53.137554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98eed5-a2f3-443b-bbfe-17a7123fc763 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:53.137823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98eed5-a2f3-443b-bbfe-17a7123fc763 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:53.149538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98eed5-a2f3-443b-bbfe-17a7123fc763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:53.151011 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98eed5-a2f3-443b-bbfe-17a7123fc763 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:53.164727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-ab98eed5-a2f3-443b-bbfe-17a7123fc763 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:53.164727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:53.164727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:53.164727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:53.164727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:53.164727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:53.164727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:53.164727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:53.164727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:53.164727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:53.164727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:53.164727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:53.164727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:49:53.164727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:49:53.164727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:49:53.164727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:49:53.165931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:53.165931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:53.165931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:53.165931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:53.165931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:53.165931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:53.165931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:53.165931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:53.165931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:53.165931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:53.165931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:53.165931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:53.165931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:53.165931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:53.165931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:53.165931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:53.165931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:49:53.166898 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:49:53.166898 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:53.166898 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:49:53.166898 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 633/1269] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:53.172519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1967bc-a7b8-4259-ad4c-9d754a56e7ae None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:53.278490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1967bc-a7b8-4259-ad4c-9d754a56e7ae None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:53.317390 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d060b-7b7f-4a66-8b6e-6eb7326a7a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:53.318145 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d060b-7b7f-4a66-8b6e-6eb7326a7a31 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:53.318434 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d060b-7b7f-4a66-8b6e-6eb7326a7a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:53.318712 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d060b-7b7f-4a66-8b6e-6eb7326a7a31 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:53.319365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d060b-7b7f-4a66-8b6e-6eb7326a7a31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:53.319670 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d060b-7b7f-4a66-8b6e-6eb7326a7a31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:53.321049 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d060b-7b7f-4a66-8b6e-6eb7326a7a31 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:53.333801 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 637/1270] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:53.339734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98eed5-a2f3-443b-bbfe-17a7123fc763 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:53.372600 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:53.450701 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98eed5-a2f3-443b-bbfe-17a7123fc763 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:53.487099 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751f614-4442-474c-a0a6-b82723932426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:53.487733 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751f614-4442-474c-a0a6-b82723932426 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:53.488016 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751f614-4442-474c-a0a6-b82723932426 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:53.488297 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751f614-4442-474c-a0a6-b82723932426 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:53.495252 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751f614-4442-474c-a0a6-b82723932426 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:53.497206 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751f614-4442-474c-a0a6-b82723932426 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:53.508534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-a751f614-4442-474c-a0a6-b82723932426 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:49:53.508534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:53.508534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:53.508534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:53.508534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:53.508534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:53.508534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:53.508534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:53.508534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:53.508534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:53.508534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:53.508534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:53.508534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:49:53.508534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:49:53.508534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:49:53.509635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:49:53.509635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:53.509635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:53.509635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:53.509635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:53.509635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:53.509635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:53.509635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:53.509635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:53.509635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:53.509635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:53.509635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:53.509635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:53.509635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:53.509635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:53.509635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:53.509635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:53.510788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:49:53.510788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:49:53.510788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:49:53.510788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:49:53.510788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:49:53.510788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:49:53.510788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:49:53.510788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:49:53.510788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 634/1271] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:49:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:53.516201 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d060b-7b7f-4a66-8b6e-6eb7326a7a31 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:53.577252 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:53.621254 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d060b-7b7f-4a66-8b6e-6eb7326a7a31 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:53.656771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d17cea-9c40-49f2-9596-7add53ca2072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:53.657384 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d17cea-9c40-49f2-9596-7add53ca2072 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:53.657675 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d17cea-9c40-49f2-9596-7add53ca2072 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:53.657985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d17cea-9c40-49f2-9596-7add53ca2072 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:53.658684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d17cea-9c40-49f2-9596-7add53ca2072 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:53.658994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d17cea-9c40-49f2-9596-7add53ca2072 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:53.660435 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d17cea-9c40-49f2-9596-7add53ca2072 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:53.673970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 638/1272] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:49:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:53.680084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751f614-4442-474c-a0a6-b82723932426 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:53.777498 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:53.786385 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751f614-4442-474c-a0a6-b82723932426 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:53.821889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd54077-c947-4bb7-9fd1-b4ae8ab470a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:53.822579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd54077-c947-4bb7-9fd1-b4ae8ab470a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:53.822850 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd54077-c947-4bb7-9fd1-b4ae8ab470a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:53.823130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd54077-c947-4bb7-9fd1-b4ae8ab470a6 None admin] PATH_INFO: `/v3/projects/449a5fac1acd478abea544446c08ecd9/users/0ca10dd7ed714edb8b87017deecc41b6/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:53.825031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd54077-c947-4bb7-9fd1-b4ae8ab470a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=449a5fac1acd478abea544446c08ecd9, user_id=0ca10dd7ed714edb8b87017deecc41b6, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:53.826623 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd54077-c947-4bb7-9fd1-b4ae8ab470a6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:53.919052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 635/1273] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 14 11:49:53 2022] PUT /identity/v3/projects/449a5fac1acd478abea544446c08ecd9/users/0ca10dd7ed714edb8b87017deecc41b6/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 11:49:56.094586 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e989a-b2fc-4e28-893d-410da0981f86 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:56.095524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e989a-b2fc-4e28-893d-410da0981f86 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:56.095524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e989a-b2fc-4e28-893d-410da0981f86 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:56.097064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 639/1274] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:49:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:49:56.102998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d47223-564a-4931-903a-b3879d6d54ac None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:56.104421 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d47223-564a-4931-903a-b3879d6d54ac None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:56.104421 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d47223-564a-4931-903a-b3879d6d54ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:56.121958 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-48d47223-564a-4931-903a-b3879d6d54ac None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:49:56.127214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-48d47223-564a-4931-903a-b3879d6d54ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:49:56.191695 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 636/1275] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:49:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 11:49:56.465023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e989a-b2fc-4e28-893d-410da0981f86 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:56.571157 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e989a-b2fc-4e28-893d-410da0981f86 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:56.590613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:56.607389 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390a003-3aa9-4bb2-b8c5-3d2cf38572d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:56.608030 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390a003-3aa9-4bb2-b8c5-3d2cf38572d0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:56.608315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390a003-3aa9-4bb2-b8c5-3d2cf38572d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:56.608588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390a003-3aa9-4bb2-b8c5-3d2cf38572d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:56.619308 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390a003-3aa9-4bb2-b8c5-3d2cf38572d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 11:49:56.624794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-a390a003-3aa9-4bb2-b8c5-3d2cf38572d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:49:56.624794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:56.624794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:56.624794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:56.624794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:56.624794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:56.624794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:56.624794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:56.624794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:56.624794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:56.624794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:56.624794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:56.624794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 11:49:56.624794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 11:49:56.624794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 11:49:56.624794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 11:49:56.627125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 11:49:56.627125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 11:49:56.627125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 11:49:56.627125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 11:49:56.627125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:56.627125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:56.627125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:56.627125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:56.627125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:56.627125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:56.627125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:56.627125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:56.627125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:56.627125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:56.627125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:56.627125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:56.628262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:56.628262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:56.628262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:56.628262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:56.628262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 11:49:56.628262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 11:49:56.628262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 11:49:56.628262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 11:49:56.628262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 11:49:56.628262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 11:49:56.628262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 11:49:56.628262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:49:56.628262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 640/1276] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 14 11:49:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:56.632982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d47223-564a-4931-903a-b3879d6d54ac None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:56.725179 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:56.744886 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d47223-564a-4931-903a-b3879d6d54ac None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:56.781822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862841fe-77e9-485b-9ac8-c32bcbc06e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:56.782463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862841fe-77e9-485b-9ac8-c32bcbc06e70 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:56.782744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862841fe-77e9-485b-9ac8-c32bcbc06e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:56.783018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862841fe-77e9-485b-9ac8-c32bcbc06e70 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:56.783681 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862841fe-77e9-485b-9ac8-c32bcbc06e70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:56.783982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862841fe-77e9-485b-9ac8-c32bcbc06e70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:56.785442 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862841fe-77e9-485b-9ac8-c32bcbc06e70 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:56.795334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 637/1277] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 14 11:49:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 11:49:56.802915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390a003-3aa9-4bb2-b8c5-3d2cf38572d0 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:56.930194 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390a003-3aa9-4bb2-b8c5-3d2cf38572d0 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:56.969496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968af1cf-d2e3-4146-8045-8ab5226f1a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:56.970184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968af1cf-d2e3-4146-8045-8ab5226f1a84 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:56.970465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968af1cf-d2e3-4146-8045-8ab5226f1a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:56.970740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968af1cf-d2e3-4146-8045-8ab5226f1a84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:56.985131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968af1cf-d2e3-4146-8045-8ab5226f1a84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:56.985131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968af1cf-d2e3-4146-8045-8ab5226f1a84 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:57.001364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-968af1cf-d2e3-4146-8045-8ab5226f1a84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:57.001364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:57.001364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:57.001364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:57.001364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:57.001364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:57.001364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:57.001364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:57.001364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:57.001364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:57.001364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:57.001364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:57.001364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:49:57.001364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:49:57.001364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:49:57.001364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:49:57.002580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:57.002580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:57.002580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:57.002580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:57.002580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:57.002580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:57.002580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:57.002580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:57.002580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:57.002580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:57.002580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:57.002580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:57.002580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:57.002580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:57.002580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:57.002580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:57.002580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:49:57.003499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:49:57.003499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:57.003499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:49:57.003499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 641/1278] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:57.009307 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862841fe-77e9-485b-9ac8-c32bcbc06e70 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:57.115829 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862841fe-77e9-485b-9ac8-c32bcbc06e70 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:57.157923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876363e-0974-4972-b47c-6840f719c80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:57.158761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876363e-0974-4972-b47c-6840f719c80f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:57.159191 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876363e-0974-4972-b47c-6840f719c80f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:57.159609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876363e-0974-4972-b47c-6840f719c80f None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:57.160430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876363e-0974-4972-b47c-6840f719c80f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:57.160887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876363e-0974-4972-b47c-6840f719c80f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:57.162575 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876363e-0974-4972-b47c-6840f719c80f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:57.177888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 638/1279] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:57.184874 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968af1cf-d2e3-4146-8045-8ab5226f1a84 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:57.218571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:57.290258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968af1cf-d2e3-4146-8045-8ab5226f1a84 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:57.325512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc1ec4-c35d-4420-9939-c3412d406980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:57.326176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc1ec4-c35d-4420-9939-c3412d406980 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:57.326457 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc1ec4-c35d-4420-9939-c3412d406980 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:57.326761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc1ec4-c35d-4420-9939-c3412d406980 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:57.334851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc1ec4-c35d-4420-9939-c3412d406980 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:57.336336 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc1ec4-c35d-4420-9939-c3412d406980 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:57.348804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-3ebc1ec4-c35d-4420-9939-c3412d406980 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 11:49:57.348804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:57.348804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:57.348804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:57.348804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:57.348804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:57.348804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:57.348804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:57.348804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:57.348804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:57.348804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:57.348804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:57.348804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 11:49:57.348804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 11:49:57.348804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 11:49:57.348804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 11:49:57.350105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:57.350105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:57.350105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 11:49:57.350105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:49:57.350105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 11:49:57.350105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 11:49:57.350105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:57.350105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:57.350105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:57.350105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:57.350105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:57.350105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:57.350105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:57.350105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:57.350105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:57.350105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:57.350105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:57.350105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:57.351132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:57.351132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:57.351132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 11:49:57.351132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 11:49:57.351132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 11:49:57.351132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 11:49:57.351132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 11:49:57.351132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 11:49:57.351132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 11:49:57.351132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:49:57.351132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 642/1280] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:57.356702 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876363e-0974-4972-b47c-6840f719c80f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:57.361462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:57.470305 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876363e-0974-4972-b47c-6840f719c80f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:57.508522 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27d16a-00e5-4c63-afcb-b78e19b04322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:57.509211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27d16a-00e5-4c63-afcb-b78e19b04322 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:57.509497 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27d16a-00e5-4c63-afcb-b78e19b04322 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:57.509775 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27d16a-00e5-4c63-afcb-b78e19b04322 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:57.510587 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27d16a-00e5-4c63-afcb-b78e19b04322 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:57.510897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27d16a-00e5-4c63-afcb-b78e19b04322 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:57.512519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27d16a-00e5-4c63-afcb-b78e19b04322 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:57.534914 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 639/1281] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Sep 14 11:49:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:57.541429 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc1ec4-c35d-4420-9939-c3412d406980 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:57.663243 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:57.675644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc1ec4-c35d-4420-9939-c3412d406980 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:57.724964 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8113c-e53b-4e3c-a753-089dc5b69a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:57.725541 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8113c-e53b-4e3c-a753-089dc5b69a51 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:57.725825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8113c-e53b-4e3c-a753-089dc5b69a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:57.726130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8113c-e53b-4e3c-a753-089dc5b69a51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:57.822466 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8113c-e53b-4e3c-a753-089dc5b69a51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:57.824182 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8113c-e53b-4e3c-a753-089dc5b69a51 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:57.927931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:57.940327 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 643/1282] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 14 11:49:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 11:49:57.946440 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27d16a-00e5-4c63-afcb-b78e19b04322 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:58.027575 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:58.078426 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27d16a-00e5-4c63-afcb-b78e19b04322 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:58.118210 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1179b37-8eb1-49e9-9bef-9039983d2411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:58.118913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1179b37-8eb1-49e9-9bef-9039983d2411 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:58.119113 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1179b37-8eb1-49e9-9bef-9039983d2411 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:58.119386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1179b37-8eb1-49e9-9bef-9039983d2411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:58.133769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1179b37-8eb1-49e9-9bef-9039983d2411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:58.135399 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1179b37-8eb1-49e9-9bef-9039983d2411 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:58.151319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-c1179b37-8eb1-49e9-9bef-9039983d2411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:58.151319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:58.151319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:58.151319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:58.151319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:58.151319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:58.151319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:58.151319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:58.151319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:58.151319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:58.151319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:58.151319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:58.151319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 11:49:58.151319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 11:49:58.151319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 11:49:58.151319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 11:49:58.152609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:58.152609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:58.152609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:58.152609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:58.152609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:58.152609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:58.152609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:58.152609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:58.152609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:58.152609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:58.152609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:58.152609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:58.152609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:58.152609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:58.152609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:58.152609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:58.152609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 11:49:58.153774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 11:49:58.153774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 11:49:58.153774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:49:58.154358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 640/1283] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 11:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:58.158405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8113c-e53b-4e3c-a753-089dc5b69a51 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:58.267296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8113c-e53b-4e3c-a753-089dc5b69a51 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:58.303041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291aa84c-c2bf-4952-bbe2-0f0476e2acc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:58.303644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291aa84c-c2bf-4952-bbe2-0f0476e2acc0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:58.303924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291aa84c-c2bf-4952-bbe2-0f0476e2acc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:58.304205 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291aa84c-c2bf-4952-bbe2-0f0476e2acc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:58.304890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291aa84c-c2bf-4952-bbe2-0f0476e2acc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:58.305193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291aa84c-c2bf-4952-bbe2-0f0476e2acc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:58.306616 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291aa84c-c2bf-4952-bbe2-0f0476e2acc0 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:58.318920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 644/1284] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 14 11:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:58.324753 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1179b37-8eb1-49e9-9bef-9039983d2411 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:58.439917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1179b37-8eb1-49e9-9bef-9039983d2411 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:58.454478 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:58.474907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a35120-f404-4909-b52c-c58e05e53777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:58.475542 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a35120-f404-4909-b52c-c58e05e53777 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:58.475823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a35120-f404-4909-b52c-c58e05e53777 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:58.476160 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a35120-f404-4909-b52c-c58e05e53777 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:58.485475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a35120-f404-4909-b52c-c58e05e53777 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:58.485475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a35120-f404-4909-b52c-c58e05e53777 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:58.498012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-92a35120-f404-4909-b52c-c58e05e53777 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:49:58.498012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:49:58.498012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:49:58.498012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:49:58.498012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:49:58.498012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:49:58.498012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:49:58.498012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:49:58.498012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:49:58.498012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:49:58.498012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:49:58.498012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:49:58.498012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 11:49:58.498012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 11:49:58.498012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 11:49:58.499240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 11:49:58.499240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:49:58.499240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:49:58.499240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:49:58.499240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:49:58.499240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:49:58.499240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:49:58.499240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:49:58.499240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Sep 14 11:49:58.499240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:49:58.499240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:49:58.499240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:49:58.499240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:49:58.499240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:49:58.499240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:49:58.499240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:49:58.499240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:49:58.500463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 11:49:58.500463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 11:49:58.500463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 11:49:58.500463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 11:49:58.500463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 11:49:58.500463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 11:49:58.500463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 11:49:58.500463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Sep 14 11:49:58.500463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 641/1285] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 14 11:49:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:49:58.505439 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291aa84c-c2bf-4952-bbe2-0f0476e2acc0 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:58.560519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:49:58.613781 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291aa84c-c2bf-4952-bbe2-0f0476e2acc0 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:58.651919 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be81fa3-0215-4856-bb94-83337820608b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:58.652559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be81fa3-0215-4856-bb94-83337820608b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:58.652841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be81fa3-0215-4856-bb94-83337820608b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:58.653118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be81fa3-0215-4856-bb94-83337820608b None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:58.653929 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be81fa3-0215-4856-bb94-83337820608b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:58.665634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be81fa3-0215-4856-bb94-83337820608b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:58.665634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be81fa3-0215-4856-bb94-83337820608b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:58.670929 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 645/1286] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 14 11:49:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:49:58.677732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a35120-f404-4909-b52c-c58e05e53777 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:49:58.780147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a35120-f404-4909-b52c-c58e05e53777 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:49:58.814079 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbc36f-fca4-49ae-98f5-23bb2dd9de96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:49:58.814759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbc36f-fca4-49ae-98f5-23bb2dd9de96 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:49:58.815037 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbc36f-fca4-49ae-98f5-23bb2dd9de96 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:49:58.815306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbc36f-fca4-49ae-98f5-23bb2dd9de96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:49:58.816025 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbc36f-fca4-49ae-98f5-23bb2dd9de96 None admin] RBAC: Adding query filter params (role.id=4a031234e1ef4fdf8ce238fbdd96fc7d, scope.project.id=449a5fac1acd478abea544446c08ecd9, user.id=0ca10dd7ed714edb8b87017deecc41b6) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:49:58.816330 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbc36f-fca4-49ae-98f5-23bb2dd9de96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:49:58.817750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbc36f-fca4-49ae-98f5-23bb2dd9de96 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:49:58.824327 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 642/1287] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 14 11:49:58 2022] GET /identity/v3/role_assignments?user.id=0ca10dd7ed714edb8b87017deecc41b6&scope.project.id=449a5fac1acd478abea544446c08ecd9&role.id=4a031234e1ef4fdf8ce238fbdd96fc7d => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:50:01.038541 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4fa018-beac-4050-a8dd-f20b6d998701 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:01.039081 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4fa018-beac-4050-a8dd-f20b6d998701 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:01.039196 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4fa018-beac-4050-a8dd-f20b6d998701 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:01.040656 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 646/1288] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:50:01.047274 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42232b-2226-4b52-9159-3ebacc234386 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:01.047807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42232b-2226-4b52-9159-3ebacc234386 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:01.047807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42232b-2226-4b52-9159-3ebacc234386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:01.066971 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6b42232b-2226-4b52-9159-3ebacc234386 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:50:01.069536 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6b42232b-2226-4b52-9159-3ebacc234386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:50:01.102601 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:50:01.147938 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 643/1289] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:50:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:50:01.417047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4fa018-beac-4050-a8dd-f20b6d998701 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:50:01.534622 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4fa018-beac-4050-a8dd-f20b6d998701 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:50:01.546241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:50:01.569979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb917235-6b6a-422e-9289-a28f4208bf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:50:01.570614 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb917235-6b6a-422e-9289-a28f4208bf62 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:01.570895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb917235-6b6a-422e-9289-a28f4208bf62 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:01.571212 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb917235-6b6a-422e-9289-a28f4208bf62 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:01.571794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb917235-6b6a-422e-9289-a28f4208bf62 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:50:01.573155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb917235-6b6a-422e-9289-a28f4208bf62 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:50:01.583014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-bb917235-6b6a-422e-9289-a28f4208bf62 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 11:50:01.583014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:50:01.583014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:50:01.583014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:50:01.583014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:50:01.583014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:50:01.583014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:50:01.583014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:50:01.583014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 11:50:01.583014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:50:01.583014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:50:01.583014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:50:01.583014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:50:01.583014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:50:01.583014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:50:01.583014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:50:01.583014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:50:01.586795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:50:01.586795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:50:01.586795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:50:01.586795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:50:01.586795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:50:01.586795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:50:01.586795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:50:01.586795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:50:01.586795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:50:01.586795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:50:01.586795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:50:01.586795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:50:01.586795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:50:01.586795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:50:01.586795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:50:01.586795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:50:01.588178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:50:01.588178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:50:01.588178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:50:01.588178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:50:01.588178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:50:01.588178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:50:01.588178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:50:01.588178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:50:01.588178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:50:01.588178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:50:01.588178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:50:01.588178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:50:01.588178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 11:50:01.588178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:50:01.588178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 647/1290] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 14 11:50:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:50:01.592794 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42232b-2226-4b52-9159-3ebacc234386 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:50:01.698177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42232b-2226-4b52-9159-3ebacc234386 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:50:01.733578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb0bab-d69a-4f77-a897-ff56c84e8798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:50:01.734221 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb0bab-d69a-4f77-a897-ff56c84e8798 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:01.734550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb0bab-d69a-4f77-a897-ff56c84e8798 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:01.734805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb0bab-d69a-4f77-a897-ff56c84e8798 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:01.735435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb0bab-d69a-4f77-a897-ff56c84e8798 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:50:01.735752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb0bab-d69a-4f77-a897-ff56c84e8798 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:50:01.737097 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb0bab-d69a-4f77-a897-ff56c84e8798 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:50:01.742004 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 644/1291] 10.222.0.62 () {56 vars in 1116 bytes} [Wed Sep 14 11:50:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:50:01.748345 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb917235-6b6a-422e-9289-a28f4208bf62 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:50:01.852883 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb917235-6b6a-422e-9289-a28f4208bf62 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:50:01.888678 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213986f1-6930-4e39-848a-8514dc991ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:50:01.889392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213986f1-6930-4e39-848a-8514dc991ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:01.889676 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213986f1-6930-4e39-848a-8514dc991ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:01.889984 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213986f1-6930-4e39-848a-8514dc991ad3 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:01.890652 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213986f1-6930-4e39-848a-8514dc991ad3 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:50:01.891044 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213986f1-6930-4e39-848a-8514dc991ad3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:50:01.892440 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213986f1-6930-4e39-848a-8514dc991ad3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:50:01.897321 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 648/1292] 10.222.0.62 () {56 vars in 1116 bytes} [Wed Sep 14 11:50:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:50:04.140582 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f9e91-c329-4c9d-9785-5a979c863080 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:04.141172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f9e91-c329-4c9d-9785-5a979c863080 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:04.141172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f9e91-c329-4c9d-9785-5a979c863080 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:04.142504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 645/1293] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:50:04.147867 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d447924-42e6-42b1-817f-e5488cd1c205 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:04.148320 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d447924-42e6-42b1-817f-e5488cd1c205 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:04.148431 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d447924-42e6-42b1-817f-e5488cd1c205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:04.167412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3d447924-42e6-42b1-817f-e5488cd1c205 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:50:04.169549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3d447924-42e6-42b1-817f-e5488cd1c205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:50:04.199545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:50:04.237322 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 649/1294] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:50:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:50:04.505710 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f9e91-c329-4c9d-9785-5a979c863080 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:50:04.538636 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:50:04.609708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f9e91-c329-4c9d-9785-5a979c863080 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:50:04.643272 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2851047-56d5-45a4-b662-1a2f7ff5c6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:50:04.643997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2851047-56d5-45a4-b662-1a2f7ff5c6e4 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:04.644279 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2851047-56d5-45a4-b662-1a2f7ff5c6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:04.644549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2851047-56d5-45a4-b662-1a2f7ff5c6e4 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:04.645208 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2851047-56d5-45a4-b662-1a2f7ff5c6e4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:50:04.646581 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2851047-56d5-45a4-b662-1a2f7ff5c6e4 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:50:04.731400 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 646/1295] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 14 11:50:04 2022] POST /identity/v3/services => generated 246 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:50:06.986094 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb29284-07b1-4a3e-a9a6-9473eb6df8b5 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:06.986628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb29284-07b1-4a3e-a9a6-9473eb6df8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:06.986746 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb29284-07b1-4a3e-a9a6-9473eb6df8b5 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:06.988009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 650/1296] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:50:06.993526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2c253-aa03-4206-9233-42cf9c645c1e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:06.994011 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2c253-aa03-4206-9233-42cf9c645c1e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:06.994111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2c253-aa03-4206-9233-42cf9c645c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:07.013547 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-24f2c253-aa03-4206-9233-42cf9c645c1e None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:50:07.014784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-24f2c253-aa03-4206-9233-42cf9c645c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:50:07.087645 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 647/1297] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:50:06 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:50:07.355940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb29284-07b1-4a3e-a9a6-9473eb6df8b5 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:50:07.460315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb29284-07b1-4a3e-a9a6-9473eb6df8b5 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:50:07.495220 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123bf99-40b8-4712-b114-97a96ca4ccdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:50:07.495901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123bf99-40b8-4712-b114-97a96ca4ccdd None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:07.496674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123bf99-40b8-4712-b114-97a96ca4ccdd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:07.496960 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123bf99-40b8-4712-b114-97a96ca4ccdd None admin] PATH_INFO: `/v3/services/placement` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:07.497555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123bf99-40b8-4712-b114-97a96ca4ccdd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:50:07.499006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123bf99-40b8-4712-b114-97a96ca4ccdd None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:50:07.508339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-0123bf99-40b8-4712-b114-97a96ca4ccdd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 11:50:07.508339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:50:07.508339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:50:07.508339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:50:07.508339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:50:07.508339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:50:07.508339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:50:07.508339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:50:07.508339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 11:50:07.508339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:50:07.508339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:50:07.508339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:50:07.508339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:50:07.508339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:50:07.508339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:50:07.508339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:50:07.508339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:50:07.510303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:50:07.510303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:50:07.510303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:50:07.510303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:50:07.510303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:50:07.510303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:50:07.510303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:50:07.510303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:50:07.510303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:50:07.510303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:50:07.510303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:50:07.510303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:50:07.510303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:50:07.510303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:50:07.510303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:50:07.510303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:50:07.511711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:50:07.511711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:50:07.511711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:50:07.511711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:50:07.511711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:50:07.511711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:50:07.511711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:50:07.511711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:50:07.511711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:50:07.511711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:50:07.511711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:50:07.511711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:50:07.511711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 11:50:07.511711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:50:07.514713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 651/1298] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 14 11:50:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:50:07.515699 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2c253-aa03-4206-9233-42cf9c645c1e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:50:07.534934 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:50:07.618583 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2c253-aa03-4206-9233-42cf9c645c1e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:50:07.653934 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025dcf1-7648-49ff-89f6-8f9f56590c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:50:07.654540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025dcf1-7648-49ff-89f6-8f9f56590c4e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:07.654822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025dcf1-7648-49ff-89f6-8f9f56590c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:07.655122 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025dcf1-7648-49ff-89f6-8f9f56590c4e None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:07.655755 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025dcf1-7648-49ff-89f6-8f9f56590c4e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:50:07.656157 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025dcf1-7648-49ff-89f6-8f9f56590c4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:50:07.657529 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025dcf1-7648-49ff-89f6-8f9f56590c4e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:50:07.662928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 648/1299] 10.222.0.62 () {56 vars in 1116 bytes} [Wed Sep 14 11:50:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:50:07.669843 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123bf99-40b8-4712-b114-97a96ca4ccdd None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:50:07.702829 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:50:07.773129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123bf99-40b8-4712-b114-97a96ca4ccdd None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:50:07.808678 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7893304-cc10-49f7-9630-f6efbf19ec01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:50:07.809289 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7893304-cc10-49f7-9630-f6efbf19ec01 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:07.809575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7893304-cc10-49f7-9630-f6efbf19ec01 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:07.809882 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7893304-cc10-49f7-9630-f6efbf19ec01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:07.810596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7893304-cc10-49f7-9630-f6efbf19ec01 None admin] RBAC: Adding query filter params (interface=public, service_id=e594f90a09f4459694c0232e8c306cb5, region_id=RegionOne) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:50:07.810908 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7893304-cc10-49f7-9630-f6efbf19ec01 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:50:07.812278 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7893304-cc10-49f7-9630-f6efbf19ec01 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:50:07.817548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 652/1300] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Sep 14 11:50:07 2022] GET /identity/v3/endpoints?service_id=e594f90a09f4459694c0232e8c306cb5&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:50:07.823714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025dcf1-7648-49ff-89f6-8f9f56590c4e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:50:07.927491 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:50:07.945908 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025dcf1-7648-49ff-89f6-8f9f56590c4e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:50:07.982827 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ef973-7648-4953-850b-35aadb113d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:50:07.983440 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ef973-7648-4953-850b-35aadb113d30 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:07.983722 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ef973-7648-4953-850b-35aadb113d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:07.983995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ef973-7648-4953-850b-35aadb113d30 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:07.984602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ef973-7648-4953-850b-35aadb113d30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:50:07.986145 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ef973-7648-4953-850b-35aadb113d30 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:50:07.993057 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 649/1301] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Sep 14 11:50:07 2022] GET /identity/v3/services => generated 2191 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 11:50:10.149822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1344a5d-9e95-4245-ae04-a3eab2314ed4 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:10.150395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1344a5d-9e95-4245-ae04-a3eab2314ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:10.150395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1344a5d-9e95-4245-ae04-a3eab2314ed4 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:10.151751 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 653/1302] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:50:10.157216 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c573c4-d16e-4aa7-88fd-d9786907ecbb None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:10.161616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c573c4-d16e-4aa7-88fd-d9786907ecbb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:10.161616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c573c4-d16e-4aa7-88fd-d9786907ecbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:10.176954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a4c573c4-d16e-4aa7-88fd-d9786907ecbb None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:50:10.178175 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a4c573c4-d16e-4aa7-88fd-d9786907ecbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:50:10.243724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 650/1303] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:50:10 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 11:50:10.512766 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1344a5d-9e95-4245-ae04-a3eab2314ed4 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:50:10.611899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:50:10.618707 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1344a5d-9e95-4245-ae04-a3eab2314ed4 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:50:10.653820 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfa9f8-29f1-4962-8420-907876ad000c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:50:10.654572 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfa9f8-29f1-4962-8420-907876ad000c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:10.654775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfa9f8-29f1-4962-8420-907876ad000c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:10.655054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfa9f8-29f1-4962-8420-907876ad000c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:10.655646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfa9f8-29f1-4962-8420-907876ad000c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:50:10.657027 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfa9f8-29f1-4962-8420-907876ad000c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:50:10.666258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-bdcfa9f8-29f1-4962-8420-907876ad000c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 11:50:10.666258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:50:10.666258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 11:50:10.666258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 11:50:10.666258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 11:50:10.666258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 11:50:10.666258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 11:50:10.666258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:50:10.666258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 11:50:10.666258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:50:10.666258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 11:50:10.666258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:50:10.666258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 11:50:10.666258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 11:50:10.666258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 11:50:10.666258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 11:50:10.666258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 11:50:10.667474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 11:50:10.667474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 11:50:10.667474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 11:50:10.667474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 11:50:10.667474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 11:50:10.667474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 11:50:10.667474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 11:50:10.667474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 11:50:10.667474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 11:50:10.667474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 11:50:10.667474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 11:50:10.667474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 11:50:10.667474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 11:50:10.667474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 11:50:10.667474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 11:50:10.667474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 11:50:10.669009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 11:50:10.669009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Sep 14 11:50:10.669009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 11:50:10.669009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Sep 14 11:50:10.669009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 11:50:10.669009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 11:50:10.669009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 11:50:10.669009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Sep 14 11:50:10.669009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 11:50:10.669009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Sep 14 11:50:10.669009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 11:50:10.669009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 11:50:10.669009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 11:50:10.669009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Sep 14 11:50:10.669009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 654/1304] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 14 11:50:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 11:50:10.674462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c573c4-d16e-4aa7-88fd-d9786907ecbb None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:50:10.755044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:50:10.778313 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c573c4-d16e-4aa7-88fd-d9786907ecbb None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:50:10.813851 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb1f6a-7184-40c9-a598-c64a00c3f40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:50:10.814676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb1f6a-7184-40c9-a598-c64a00c3f40f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:10.814960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb1f6a-7184-40c9-a598-c64a00c3f40f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:10.815233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb1f6a-7184-40c9-a598-c64a00c3f40f None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:10.815883 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb1f6a-7184-40c9-a598-c64a00c3f40f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 11:50:10.816370 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb1f6a-7184-40c9-a598-c64a00c3f40f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:50:10.818302 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb1f6a-7184-40c9-a598-c64a00c3f40f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:50:10.823765 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 651/1305] 10.222.0.62 () {56 vars in 1116 bytes} [Wed Sep 14 11:50:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 11:50:10.829822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfa9f8-29f1-4962-8420-907876ad000c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:50:10.935905 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfa9f8-29f1-4962-8420-907876ad000c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:50:10.970540 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d10678-9f24-4c14-9eb2-3bf923d6ddf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:50:10.971687 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d10678-9f24-4c14-9eb2-3bf923d6ddf9 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:50:10.971687 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d10678-9f24-4c14-9eb2-3bf923d6ddf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:50:10.971890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d10678-9f24-4c14-9eb2-3bf923d6ddf9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:50:10.972393 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d10678-9f24-4c14-9eb2-3bf923d6ddf9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:50:10.973770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d10678-9f24-4c14-9eb2-3bf923d6ddf9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:50:11.057038 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 655/1306] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 11:50:10 2022] POST /identity/v3/endpoints => generated 331 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 11:50:20.820525 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=30 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:50:21.051974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:16.173738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94160f6-f0ec-4f62-b9cd-11ce4302ffbc None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:16.174816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94160f6-f0ec-4f62-b9cd-11ce4302ffbc None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:16.175393 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94160f6-f0ec-4f62-b9cd-11ce4302ffbc None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:16.178238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 652/1307] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:57:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:57:16.184440 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba337b-6de3-4524-86f7-38c7166d918e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:16.184909 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba337b-6de3-4524-86f7-38c7166d918e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:16.185003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba337b-6de3-4524-86f7-38c7166d918e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:16.315445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-20ba337b-6de3-4524-86f7-38c7166d918e None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:57:16.316679 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-20ba337b-6de3-4524-86f7-38c7166d918e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:16.435228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 656/1308] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:57:16.731133 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127f63a-af21-4888-93e2-6261c058c792 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:16.731454 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127f63a-af21-4888-93e2-6261c058c792 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:16.731735 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127f63a-af21-4888-93e2-6261c058c792 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:16.733445 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 653/1309] 10.222.0.62 () {54 vars in 965 bytes} [Wed Sep 14 11:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:57:16.739716 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f45cd-d397-4bfb-83e6-4e531959a506 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:16.740001 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f45cd-d397-4bfb-83e6-4e531959a506 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:16.740286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f45cd-d397-4bfb-83e6-4e531959a506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:16.841643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9b5f45cd-d397-4bfb-83e6-4e531959a506 None None] MFA Rules not processed for user `13f1840d92d94c3ebd16000a4889c377`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:57:16.842957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9b5f45cd-d397-4bfb-83e6-4e531959a506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:16.869917 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:16.941160 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 657/1310] 10.222.0.62 () {58 vars in 1062 bytes} [Wed Sep 14 11:57:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:57:16.950463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127f63a-af21-4888-93e2-6261c058c792 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:57:16.952224 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b127f63a-af21-4888-93e2-6261c058c792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:17.094593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127f63a-af21-4888-93e2-6261c058c792 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:57:17.188635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc077a5-7a89-41f2-a57f-c2d01938c705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:57:17.189305 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc077a5-7a89-41f2-a57f-c2d01938c705 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:17.189527 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc077a5-7a89-41f2-a57f-c2d01938c705 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:17.190031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc077a5-7a89-41f2-a57f-c2d01938c705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:17.191698 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-dfc077a5-7a89-41f2-a57f-c2d01938c705 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:17.260331 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:17.345924 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc077a5-7a89-41f2-a57f-c2d01938c705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:57:17.348320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc077a5-7a89-41f2-a57f-c2d01938c705 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:57:17.370798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 654/1311] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:57:20.027788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf14755-792d-4e2b-af26-f25afcc8b011 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:20.028308 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf14755-792d-4e2b-af26-f25afcc8b011 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:20.028403 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf14755-792d-4e2b-af26-f25afcc8b011 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:20.029682 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 658/1312] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:57:20.035079 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e03cf-4f2d-40dd-bef3-b5f90873b429 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:20.035526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e03cf-4f2d-40dd-bef3-b5f90873b429 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:20.035621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e03cf-4f2d-40dd-bef3-b5f90873b429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:20.124946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-b98e03cf-4f2d-40dd-bef3-b5f90873b429 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:57:20.126189 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b98e03cf-4f2d-40dd-bef3-b5f90873b429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:20.240827 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 655/1313] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:57:20.527706 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf14755-792d-4e2b-af26-f25afcc8b011 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:57:20.529237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-caf14755-792d-4e2b-af26-f25afcc8b011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:20.634424 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf14755-792d-4e2b-af26-f25afcc8b011 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:57:20.705559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:20.707873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b9447-e575-4ac9-875e-ba6edf830de5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:57:20.708474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b9447-e575-4ac9-875e-ba6edf830de5 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:20.708754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b9447-e575-4ac9-875e-ba6edf830de5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:20.709023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b9447-e575-4ac9-875e-ba6edf830de5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:20.710662 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cb8b9447-e575-4ac9-875e-ba6edf830de5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:20.822066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b9447-e575-4ac9-875e-ba6edf830de5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:57:20.823548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b9447-e575-4ac9-875e-ba6edf830de5 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:57:20.842872 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 659/1314] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:57:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:57:23.690456 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ca096-1b3d-4b7f-b9ce-e7ff2d1f8fb7 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:23.691013 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ca096-1b3d-4b7f-b9ce-e7ff2d1f8fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:23.691013 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ca096-1b3d-4b7f-b9ce-e7ff2d1f8fb7 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:23.692354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 656/1315] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:57:23.698309 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45911d4-2494-4fbb-a372-0640f12ec287 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:23.698739 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45911d4-2494-4fbb-a372-0640f12ec287 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:23.698832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45911d4-2494-4fbb-a372-0640f12ec287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:23.768578 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f45911d4-2494-4fbb-a372-0640f12ec287 None None] MFA Rules not processed for user `d56b1b366e684a00aa34e438baa482b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:57:23.769852 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f45911d4-2494-4fbb-a372-0640f12ec287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:23.824479 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:23.873850 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 660/1316] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:57:23 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:57:24.165428 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ca096-1b3d-4b7f-b9ce-e7ff2d1f8fb7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:57:24.167593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-550ca096-1b3d-4b7f-b9ce-e7ff2d1f8fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:24.177819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:24.281887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ca096-1b3d-4b7f-b9ce-e7ff2d1f8fb7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:57:24.364080 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaae93c-4683-4519-bba6-1c7db78c69a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:57:24.364714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaae93c-4683-4519-bba6-1c7db78c69a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:24.365035 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaae93c-4683-4519-bba6-1c7db78c69a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:24.365326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaae93c-4683-4519-bba6-1c7db78c69a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:24.367113 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-dfaae93c-4683-4519-bba6-1c7db78c69a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:24.436266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:24.487331 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaae93c-4683-4519-bba6-1c7db78c69a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:57:24.488803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaae93c-4683-4519-bba6-1c7db78c69a7 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:57:24.509115 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 657/1317] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:57:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:57:27.708063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63688597-22b2-49e9-8f97-775e6523bcbf None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:27.708576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63688597-22b2-49e9-8f97-775e6523bcbf None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:27.708576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63688597-22b2-49e9-8f97-775e6523bcbf None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:27.709895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 661/1318] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:57:27.715189 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097e00e-2d89-4b18-b60e-81c8719791e6 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:27.715645 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097e00e-2d89-4b18-b60e-81c8719791e6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:27.715743 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097e00e-2d89-4b18-b60e-81c8719791e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:27.784439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4097e00e-2d89-4b18-b60e-81c8719791e6 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:57:27.785690 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4097e00e-2d89-4b18-b60e-81c8719791e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:27.887792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 658/1319] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:57:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:57:28.176652 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63688597-22b2-49e9-8f97-775e6523bcbf None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:57:28.178527 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-63688597-22b2-49e9-8f97-775e6523bcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:28.286844 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63688597-22b2-49e9-8f97-775e6523bcbf None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:57:28.360230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883fac98-437c-4c08-8852-833dd63863d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:57:28.360837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883fac98-437c-4c08-8852-833dd63863d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:28.361115 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883fac98-437c-4c08-8852-833dd63863d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:28.361382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883fac98-437c-4c08-8852-833dd63863d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:28.363017 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-883fac98-437c-4c08-8852-833dd63863d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:28.370732 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:28.476832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883fac98-437c-4c08-8852-833dd63863d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:57:28.478250 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883fac98-437c-4c08-8852-833dd63863d4 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:57:28.503878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 662/1320] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:57:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:57:31.976724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d3202-c800-4b5b-ac3b-c093c2b6ec6c None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:31.977296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d3202-c800-4b5b-ac3b-c093c2b6ec6c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:31.977296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d3202-c800-4b5b-ac3b-c093c2b6ec6c None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:31.979489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 659/1321] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:57:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:57:31.984558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309a852-b64d-4478-b5ff-6c7a513c8145 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:31.984978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309a852-b64d-4478-b5ff-6c7a513c8145 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:31.985120 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309a852-b64d-4478-b5ff-6c7a513c8145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:32.055149 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-6309a852-b64d-4478-b5ff-6c7a513c8145 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:57:32.056970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6309a852-b64d-4478-b5ff-6c7a513c8145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:32.124319 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:32.171749 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 663/1322] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:57:32.455721 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d3202-c800-4b5b-ac3b-c093c2b6ec6c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:57:32.457261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0f4d3202-c800-4b5b-ac3b-c093c2b6ec6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:32.477802 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:32.565547 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d3202-c800-4b5b-ac3b-c093c2b6ec6c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:57:32.653766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ff66d-64ee-4290-a9c1-382bf9287cef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:57:32.654428 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ff66d-64ee-4290-a9c1-382bf9287cef service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:32.654715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ff66d-64ee-4290-a9c1-382bf9287cef service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:32.654996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ff66d-64ee-4290-a9c1-382bf9287cef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:32.656637 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-752ff66d-64ee-4290-a9c1-382bf9287cef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:32.734389 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:32.775626 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ff66d-64ee-4290-a9c1-382bf9287cef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:57:32.776956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ff66d-64ee-4290-a9c1-382bf9287cef service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:57:32.796599 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 660/1323] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:57:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:57:35.052812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc30d82-97f6-482c-a413-f0d5b783705e None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:35.053393 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc30d82-97f6-482c-a413-f0d5b783705e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:35.053500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc30d82-97f6-482c-a413-f0d5b783705e None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:35.054965 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 664/1324] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:57:35.061454 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4289e8f-b683-4c7d-aaf8-f87c50ab6251 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:35.061995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4289e8f-b683-4c7d-aaf8-f87c50ab6251 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:35.062102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4289e8f-b683-4c7d-aaf8-f87c50ab6251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:35.131799 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f4289e8f-b683-4c7d-aaf8-f87c50ab6251 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:57:35.133133 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f4289e8f-b683-4c7d-aaf8-f87c50ab6251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:35.247050 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 661/1325] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:57:35.531840 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc30d82-97f6-482c-a413-f0d5b783705e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:57:35.533434 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ddc30d82-97f6-482c-a413-f0d5b783705e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:35.653945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc30d82-97f6-482c-a413-f0d5b783705e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:57:35.736069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce78b8-1846-4607-9525-2025327e7665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:57:35.736601 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce78b8-1846-4607-9525-2025327e7665 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:35.736884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce78b8-1846-4607-9525-2025327e7665 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:35.737158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce78b8-1846-4607-9525-2025327e7665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:35.738822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-77ce78b8-1846-4607-9525-2025327e7665 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:35.756766 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:35.867895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce78b8-1846-4607-9525-2025327e7665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:57:35.869432 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce78b8-1846-4607-9525-2025327e7665 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:57:35.889052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 665/1326] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:57:38.139149 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db1c24-961b-408e-aae0-c49dd71884a0 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:38.139746 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db1c24-961b-408e-aae0-c49dd71884a0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:38.139746 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db1c24-961b-408e-aae0-c49dd71884a0 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:38.141054 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 662/1327] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:57:38.146263 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81997a-3c02-425d-89a6-bae89e40bc83 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:38.146670 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81997a-3c02-425d-89a6-bae89e40bc83 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:38.146825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81997a-3c02-425d-89a6-bae89e40bc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:38.214434 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4f81997a-3c02-425d-89a6-bae89e40bc83 None None] MFA Rules not processed for user `d56b1b366e684a00aa34e438baa482b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:57:38.215575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4f81997a-3c02-425d-89a6-bae89e40bc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:38.281444 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:38.314577 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 666/1328] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:57:38 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:57:38.596357 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db1c24-961b-408e-aae0-c49dd71884a0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:57:38.598239 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f2db1c24-961b-408e-aae0-c49dd71884a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:38.633762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:38.715438 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db1c24-961b-408e-aae0-c49dd71884a0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:57:38.796111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd1195-bf2a-449b-b345-faf3f427f93a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:57:38.796718 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd1195-bf2a-449b-b345-faf3f427f93a service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:38.796998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd1195-bf2a-449b-b345-faf3f427f93a service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:38.797271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd1195-bf2a-449b-b345-faf3f427f93a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:38.799017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-37cd1195-bf2a-449b-b345-faf3f427f93a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:38.880745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:38.914335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd1195-bf2a-449b-b345-faf3f427f93a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:57:38.915799 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd1195-bf2a-449b-b345-faf3f427f93a service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:57:38.934874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 663/1329] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:57:38 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:57:41.639777 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2eda7-1586-47b7-928e-a5dbabf02f3e None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:41.640236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2eda7-1586-47b7-928e-a5dbabf02f3e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:41.640351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2eda7-1586-47b7-928e-a5dbabf02f3e None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:41.641699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 667/1330] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:57:41.647264 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a716b9-97a4-4211-9b34-fc797c8eea2c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:41.647733 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a716b9-97a4-4211-9b34-fc797c8eea2c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:41.647841 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a716b9-97a4-4211-9b34-fc797c8eea2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:41.715157 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e0a716b9-97a4-4211-9b34-fc797c8eea2c None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:57:41.716273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e0a716b9-97a4-4211-9b34-fc797c8eea2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:41.815148 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 664/1331] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:57:42.090301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2eda7-1586-47b7-928e-a5dbabf02f3e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:57:42.091850 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-eff2eda7-1586-47b7-928e-a5dbabf02f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:42.200065 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2eda7-1586-47b7-928e-a5dbabf02f3e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:57:42.275841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3fe86-2713-475f-8d1b-84158dd3c9a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:57:42.276446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3fe86-2713-475f-8d1b-84158dd3c9a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:42.276742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3fe86-2713-475f-8d1b-84158dd3c9a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:42.277029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3fe86-2713-475f-8d1b-84158dd3c9a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:42.278680 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c1c3fe86-2713-475f-8d1b-84158dd3c9a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:42.305490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:42.389244 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3fe86-2713-475f-8d1b-84158dd3c9a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:57:42.390698 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3fe86-2713-475f-8d1b-84158dd3c9a1 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:57:42.408982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 668/1332] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:57:44.623364 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838cace-66e2-44b4-a4bc-e67b309848c6 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:44.623951 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838cace-66e2-44b4-a4bc-e67b309848c6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:44.623951 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838cace-66e2-44b4-a4bc-e67b309848c6 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:44.625402 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 665/1333] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:57:44.631128 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4bc39-21fc-41e9-9d17-b0eda9661d9c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:44.631525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4bc39-21fc-41e9-9d17-b0eda9661d9c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:44.631691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4bc39-21fc-41e9-9d17-b0eda9661d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:44.703650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-49f4bc39-21fc-41e9-9d17-b0eda9661d9c None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:57:44.704899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-49f4bc39-21fc-41e9-9d17-b0eda9661d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:44.784185 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:44.806651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 669/1334] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:57:45.098940 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838cace-66e2-44b4-a4bc-e67b309848c6 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:57:45.100542 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a838cace-66e2-44b4-a4bc-e67b309848c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:45.142374 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:45.206725 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838cace-66e2-44b4-a4bc-e67b309848c6 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:57:45.283047 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecf6eb-d3f6-4aff-b7ee-1db0f90b678f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:57:45.283655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecf6eb-d3f6-4aff-b7ee-1db0f90b678f service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:45.283983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecf6eb-d3f6-4aff-b7ee-1db0f90b678f service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:45.284248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecf6eb-d3f6-4aff-b7ee-1db0f90b678f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:45.285865 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-41ecf6eb-d3f6-4aff-b7ee-1db0f90b678f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:45.373148 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:45.401058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ecf6eb-d3f6-4aff-b7ee-1db0f90b678f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:57:45.402543 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ecf6eb-d3f6-4aff-b7ee-1db0f90b678f service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:57:45.421799 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 666/1335] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:57:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:57:48.302754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad15677-be4c-44f5-8a1c-388812ea33f5 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:48.303323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad15677-be4c-44f5-8a1c-388812ea33f5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:48.303323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad15677-be4c-44f5-8a1c-388812ea33f5 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:48.304693 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 670/1336] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:57:48.311084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028863ed-0237-4296-a9df-2a46e8ec6775 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:48.311536 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028863ed-0237-4296-a9df-2a46e8ec6775 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:48.311642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028863ed-0237-4296-a9df-2a46e8ec6775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:48.380908 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-028863ed-0237-4296-a9df-2a46e8ec6775 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:57:48.382202 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-028863ed-0237-4296-a9df-2a46e8ec6775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:48.480320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 667/1337] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:57:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:57:48.760814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad15677-be4c-44f5-8a1c-388812ea33f5 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:57:48.762481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-aad15677-be4c-44f5-8a1c-388812ea33f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:48.870987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad15677-be4c-44f5-8a1c-388812ea33f5 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:57:48.955345 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4ee32-8f42-4674-8086-324fc3c569a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:57:48.955976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4ee32-8f42-4674-8086-324fc3c569a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:48.956237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4ee32-8f42-4674-8086-324fc3c569a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:48.956508 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4ee32-8f42-4674-8086-324fc3c569a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:48.958183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b9a4ee32-8f42-4674-8086-324fc3c569a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:48.994435 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:49.070006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4ee32-8f42-4674-8086-324fc3c569a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:57:49.072098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4ee32-8f42-4674-8086-324fc3c569a7 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:57:49.091042 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 671/1338] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:57:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:57:53.568632 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43904133-5621-4198-81ea-bdb5ed2fde07 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:53.569183 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43904133-5621-4198-81ea-bdb5ed2fde07 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:53.569183 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43904133-5621-4198-81ea-bdb5ed2fde07 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:53.570726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 668/1339] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:57:53.576217 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467b1ba-f5af-4e5c-84b8-4972e58d2b26 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:53.576754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467b1ba-f5af-4e5c-84b8-4972e58d2b26 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:53.576880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467b1ba-f5af-4e5c-84b8-4972e58d2b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:53.652889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e467b1ba-f5af-4e5c-84b8-4972e58d2b26 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:57:53.655488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e467b1ba-f5af-4e5c-84b8-4972e58d2b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:53.745574 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:53.757206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 672/1340] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:57:54.043873 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43904133-5621-4198-81ea-bdb5ed2fde07 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:57:54.046082 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-43904133-5621-4198-81ea-bdb5ed2fde07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:54.097277 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:54.155019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43904133-5621-4198-81ea-bdb5ed2fde07 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:57:54.231622 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988ac97-1c64-4c96-8a22-9fe1cf55e0d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:57:54.232230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988ac97-1c64-4c96-8a22-9fe1cf55e0d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:54.232509 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988ac97-1c64-4c96-8a22-9fe1cf55e0d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:54.232780 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988ac97-1c64-4c96-8a22-9fe1cf55e0d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:54.234407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8988ac97-1c64-4c96-8a22-9fe1cf55e0d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:54.331350 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:54.348624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988ac97-1c64-4c96-8a22-9fe1cf55e0d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:57:54.350146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988ac97-1c64-4c96-8a22-9fe1cf55e0d4 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:57:54.371008 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 669/1341] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:57:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:57:57.666303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770ea74-22fd-4b0c-b559-36a641a08803 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:57.667587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770ea74-22fd-4b0c-b559-36a641a08803 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:57.668070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770ea74-22fd-4b0c-b559-36a641a08803 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:57.669675 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 673/1342] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:57:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:57:57.676961 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f38334-d560-4ac6-b1c4-c0e13c672b00 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:57.677554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f38334-d560-4ac6-b1c4-c0e13c672b00 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:57.677824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f38334-d560-4ac6-b1c4-c0e13c672b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:57.752915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e9f38334-d560-4ac6-b1c4-c0e13c672b00 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:57:57.754082 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e9f38334-d560-4ac6-b1c4-c0e13c672b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:57.869640 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 670/1343] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:57:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:57:58.171843 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9594aee-dec5-4597-ae10-2bb94bfa14f3 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:58.172391 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9594aee-dec5-4597-ae10-2bb94bfa14f3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:58.172849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9594aee-dec5-4597-ae10-2bb94bfa14f3 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:58.174329 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 674/1344] 10.222.0.62 () {54 vars in 965 bytes} [Wed Sep 14 11:57:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:57:58.181319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55820d-33b2-4bd7-b90e-6ffad8374754 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:58.181614 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55820d-33b2-4bd7-b90e-6ffad8374754 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:58.181984 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55820d-33b2-4bd7-b90e-6ffad8374754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:58.257628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:58.268229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ad55820d-33b2-4bd7-b90e-6ffad8374754 None None] MFA Rules not processed for user `13f1840d92d94c3ebd16000a4889c377`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:57:58.268932 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ad55820d-33b2-4bd7-b90e-6ffad8374754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:58.351806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 671/1345] 10.222.0.62 () {58 vars in 1062 bytes} [Wed Sep 14 11:57:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:57:58.361144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9594aee-dec5-4597-ae10-2bb94bfa14f3 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:57:58.362689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d9594aee-dec5-4597-ae10-2bb94bfa14f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:58.508886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9594aee-dec5-4597-ae10-2bb94bfa14f3 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:57:58.597218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8142714a-9a97-43a0-81aa-8f272d6e3a94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:57:58.598015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142714a-9a97-43a0-81aa-8f272d6e3a94 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:57:58.598258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142714a-9a97-43a0-81aa-8f272d6e3a94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:57:58.598538 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142714a-9a97-43a0-81aa-8f272d6e3a94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:57:58.600372 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8142714a-9a97-43a0-81aa-8f272d6e3a94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:57:58.651659 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:57:58.726373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142714a-9a97-43a0-81aa-8f272d6e3a94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:57:58.728465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142714a-9a97-43a0-81aa-8f272d6e3a94 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:57:58.749258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 675/1346] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:57:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:58:04.050132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41097c7-d8ce-459c-b2a4-caf461344b69 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:04.050713 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41097c7-d8ce-459c-b2a4-caf461344b69 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:04.050713 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41097c7-d8ce-459c-b2a4-caf461344b69 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:04.052048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 672/1347] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:04.058513 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8103c-fe82-48d0-9ef5-3dfc6e2a66c0 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:04.059064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8103c-fe82-48d0-9ef5-3dfc6e2a66c0 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:04.059251 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8103c-fe82-48d0-9ef5-3dfc6e2a66c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:04.140446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-70d8103c-fe82-48d0-9ef5-3dfc6e2a66c0 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:04.142359 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-70d8103c-fe82-48d0-9ef5-3dfc6e2a66c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:04.282805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:04.288221 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 676/1348] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:58:04.579617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41097c7-d8ce-459c-b2a4-caf461344b69 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:58:04.581209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c41097c7-d8ce-459c-b2a4-caf461344b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:04.692942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41097c7-d8ce-459c-b2a4-caf461344b69 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:58:04.744650 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:04.771599 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf378d-a9f9-4c6e-a260-1077eed7b94c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:58:04.772212 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf378d-a9f9-4c6e-a260-1077eed7b94c service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:04.772494 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf378d-a9f9-4c6e-a260-1077eed7b94c service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:04.772774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf378d-a9f9-4c6e-a260-1077eed7b94c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:04.774926 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-11bf378d-a9f9-4c6e-a260-1077eed7b94c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:04.890342 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bf378d-a9f9-4c6e-a260-1077eed7b94c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:58:04.892042 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bf378d-a9f9-4c6e-a260-1077eed7b94c service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:58:04.914089 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 673/1349] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:58:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:58:10.192585 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1249a-2385-4ad2-9117-78cd9f948052 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:10.193679 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1249a-2385-4ad2-9117-78cd9f948052 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:10.194200 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1249a-2385-4ad2-9117-78cd9f948052 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:10.196563 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 677/1350] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 14 11:58:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:10.201838 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0505a-64f5-481b-9a53-7cd7265319f9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:10.202326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0505a-64f5-481b-9a53-7cd7265319f9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:10.202415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0505a-64f5-481b-9a53-7cd7265319f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:10.318850 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-96b0505a-64f5-481b-9a53-7cd7265319f9 None None] MFA Rules not processed for user `0ca10dd7ed714edb8b87017deecc41b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:10.318850 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-96b0505a-64f5-481b-9a53-7cd7265319f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:10.335227 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:10.481147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 674/1351] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 14 11:58:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:58:14.843069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d1337-c2be-47b5-a2f5-df02b6e28c8f None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:14.843610 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d1337-c2be-47b5-a2f5-df02b6e28c8f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:14.843610 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d1337-c2be-47b5-a2f5-df02b6e28c8f None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:14.845117 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 678/1352] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 14 11:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:14.850230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae2d6c-5715-45cb-ae34-c586737bfab2 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:14.850629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae2d6c-5715-45cb-ae34-c586737bfab2 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:14.860805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae2d6c-5715-45cb-ae34-c586737bfab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:14.932691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0aae2d6c-5715-45cb-ae34-c586737bfab2 None None] MFA Rules not processed for user `0ca10dd7ed714edb8b87017deecc41b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:14.934117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0aae2d6c-5715-45cb-ae34-c586737bfab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:15.023870 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:15.065167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 675/1353] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 14 11:58:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:58:16.327093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39ed66-ac84-48bf-a417-715a95f64d42 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:16.327093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39ed66-ac84-48bf-a417-715a95f64d42 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:16.327093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39ed66-ac84-48bf-a417-715a95f64d42 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:16.328331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 679/1354] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 14 11:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:16.333952 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9ad63-4bfb-4347-8011-0899a969793b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:16.339870 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9ad63-4bfb-4347-8011-0899a969793b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:16.339870 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9ad63-4bfb-4347-8011-0899a969793b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:16.421556 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-56a9ad63-4bfb-4347-8011-0899a969793b None None] MFA Rules not processed for user `0ca10dd7ed714edb8b87017deecc41b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:16.422769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-56a9ad63-4bfb-4347-8011-0899a969793b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:16.546200 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 676/1355] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 14 11:58:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:58:20.313842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c188ec-da0f-40b3-ae48-e33973d8b95f None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:20.315176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c188ec-da0f-40b3-ae48-e33973d8b95f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:20.315968 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c188ec-da0f-40b3-ae48-e33973d8b95f None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:20.319277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 680/1356] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:58:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:20.328505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67492d3-f241-4c16-a63a-7daeb3d00d9b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:20.329781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67492d3-f241-4c16-a63a-7daeb3d00d9b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:20.330581 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67492d3-f241-4c16-a63a-7daeb3d00d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:20.379516 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:20.412309 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a67492d3-f241-4c16-a63a-7daeb3d00d9b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:20.413581 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a67492d3-f241-4c16-a63a-7daeb3d00d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:20.533599 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 677/1357] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:58:20.845604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8e5de-a918-4e8e-861b-8986ff264bf4 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:20.846214 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8e5de-a918-4e8e-861b-8986ff264bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:20.846944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8e5de-a918-4e8e-861b-8986ff264bf4 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:20.849070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 681/1358] 10.222.0.62 () {54 vars in 962 bytes} [Wed Sep 14 11:58:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:20.856872 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e146f9-626b-4083-a71d-22294f68bb80 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:20.859252 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e146f9-626b-4083-a71d-22294f68bb80 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:20.859252 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e146f9-626b-4083-a71d-22294f68bb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:20.961411 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-28e146f9-626b-4083-a71d-22294f68bb80 None None] MFA Rules not processed for user `38b093bc4c1d4ebcbd2a6bb16b70c029`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:20.963137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-28e146f9-626b-4083-a71d-22294f68bb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:21.050440 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:21.141736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 678/1359] 10.222.0.62 () {58 vars in 1059 bytes} [Wed Sep 14 11:58:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 11:58:21.150563 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8e5de-a918-4e8e-861b-8986ff264bf4 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:58:21.152213 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c5e8e5de-a918-4e8e-861b-8986ff264bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:21.372702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8e5de-a918-4e8e-861b-8986ff264bf4 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:58:21.490877 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:21.542755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24722829-8862-4e25-bd45-09ba342381b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:58:21.543376 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24722829-8862-4e25-bd45-09ba342381b0 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:21.543650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24722829-8862-4e25-bd45-09ba342381b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:21.543920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24722829-8862-4e25-bd45-09ba342381b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:21.545577 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-24722829-8862-4e25-bd45-09ba342381b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:21.654092 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24722829-8862-4e25-bd45-09ba342381b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:58:21.655611 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24722829-8862-4e25-bd45-09ba342381b0 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:58:21.678569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 682/1360] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:58:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:58:24.300705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d45ad3-9a6f-4a62-b37d-a04a692d07cf None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:24.301511 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d45ad3-9a6f-4a62-b37d-a04a692d07cf None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:24.301511 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d45ad3-9a6f-4a62-b37d-a04a692d07cf None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:24.302805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 679/1361] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:24.308650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd05f8-e5b6-479a-b51b-973933d34716 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:24.309114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd05f8-e5b6-479a-b51b-973933d34716 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:24.309233 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd05f8-e5b6-479a-b51b-973933d34716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:24.394406 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-eedd05f8-e5b6-479a-b51b-973933d34716 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:24.395637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-eedd05f8-e5b6-479a-b51b-973933d34716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:24.439261 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:24.506248 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 683/1362] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:58:24.814124 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d45ad3-9a6f-4a62-b37d-a04a692d07cf None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:58:24.815699 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b7d45ad3-9a6f-4a62-b37d-a04a692d07cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:25.021628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d45ad3-9a6f-4a62-b37d-a04a692d07cf None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:58:25.032188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:25.185973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df677ca0-e35e-4d36-881b-d544aba2b065 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:58:25.186587 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df677ca0-e35e-4d36-881b-d544aba2b065 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:25.186877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df677ca0-e35e-4d36-881b-d544aba2b065 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:25.187137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df677ca0-e35e-4d36-881b-d544aba2b065 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:25.188860 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-df677ca0-e35e-4d36-881b-d544aba2b065 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:25.308355 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df677ca0-e35e-4d36-881b-d544aba2b065 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:58:25.310122 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df677ca0-e35e-4d36-881b-d544aba2b065 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:58:25.332336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 680/1363] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:58:28.489368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c67c6c-8e9a-4a3e-ac03-cee7980bed34 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:28.490469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c67c6c-8e9a-4a3e-ac03-cee7980bed34 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:28.490933 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c67c6c-8e9a-4a3e-ac03-cee7980bed34 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:28.492662 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 684/1364] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:58:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:28.502054 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac3150-a106-44f3-92c2-9310269a65f0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:28.502900 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac3150-a106-44f3-92c2-9310269a65f0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:28.502900 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac3150-a106-44f3-92c2-9310269a65f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:28.522205 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:28.603510 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a8ac3150-a106-44f3-92c2-9310269a65f0 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:28.604763 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a8ac3150-a106-44f3-92c2-9310269a65f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:28.766882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 681/1365] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:58:29.074815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d52b1-e200-477f-ab90-31e8ae2d6b66 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:29.075312 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d52b1-e200-477f-ab90-31e8ae2d6b66 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:29.075754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d52b1-e200-477f-ab90-31e8ae2d6b66 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:29.079084 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 685/1366] 10.222.0.62 () {54 vars in 962 bytes} [Wed Sep 14 11:58:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:29.089601 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14d447-5063-472e-b836-1079f12990b2 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:29.090222 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14d447-5063-472e-b836-1079f12990b2 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:29.090762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14d447-5063-472e-b836-1079f12990b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:29.232860 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7c14d447-5063-472e-b836-1079f12990b2 None None] MFA Rules not processed for user `38b093bc4c1d4ebcbd2a6bb16b70c029`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:29.234312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7c14d447-5063-472e-b836-1079f12990b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:29.306894 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:29.469169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 682/1367] 10.222.0.62 () {58 vars in 1059 bytes} [Wed Sep 14 11:58:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:58:29.480726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d52b1-e200-477f-ab90-31e8ae2d6b66 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:58:29.482590 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fc3d52b1-e200-477f-ab90-31e8ae2d6b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:29.745577 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d52b1-e200-477f-ab90-31e8ae2d6b66 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:58:29.781517 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:29.930322 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145c61d-da56-4b21-98a1-e83cdab17bf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:58:29.931332 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145c61d-da56-4b21-98a1-e83cdab17bf1 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:29.931948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145c61d-da56-4b21-98a1-e83cdab17bf1 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:29.932464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145c61d-da56-4b21-98a1-e83cdab17bf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:29.934531 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f145c61d-da56-4b21-98a1-e83cdab17bf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:30.037204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dec406-04b8-406b-b17a-d44466afddba None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:30.038063 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dec406-04b8-406b-b17a-d44466afddba None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:30.038063 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dec406-04b8-406b-b17a-d44466afddba None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:30.039839 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 683/1368] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:58:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:58:30.046201 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22022c-8a9c-4b6c-abf8-0c0d8c1b49e5 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:30.046610 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22022c-8a9c-4b6c-abf8-0c0d8c1b49e5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:30.046816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22022c-8a9c-4b6c-abf8-0c0d8c1b49e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:30.104722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f145c61d-da56-4b21-98a1-e83cdab17bf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:58:30.107392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f145c61d-da56-4b21-98a1-e83cdab17bf1 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:58:30.138422 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 686/1369] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:58:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:58:30.157942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ed22022c-8a9c-4b6c-abf8-0c0d8c1b49e5 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:30.159366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ed22022c-8a9c-4b6c-abf8-0c0d8c1b49e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:30.312551 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:30.326928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 684/1370] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 11:58:30.687187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab54633-76da-4863-9894-209c38ca28db None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:30.687187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab54633-76da-4863-9894-209c38ca28db None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:30.687187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab54633-76da-4863-9894-209c38ca28db None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:30.687187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 687/1371] 10.222.0.62 () {54 vars in 964 bytes} [Wed Sep 14 11:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:30.687962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ae779-e933-49fa-808f-9014c7aac038 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:30.687962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ae779-e933-49fa-808f-9014c7aac038 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:30.687962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ae779-e933-49fa-808f-9014c7aac038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:30.776620 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0f5ae779-e933-49fa-808f-9014c7aac038 None None] MFA Rules not processed for user `d27c7ff726e443e89da94f282b5cddab`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:30.778458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0f5ae779-e933-49fa-808f-9014c7aac038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:30.887273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 685/1372] 10.222.0.62 () {58 vars in 1061 bytes} [Wed Sep 14 11:58:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:58:30.900266 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab54633-76da-4863-9894-209c38ca28db None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:58:30.901849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-dab54633-76da-4863-9894-209c38ca28db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:30.945124 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:31.077628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab54633-76da-4863-9894-209c38ca28db None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:58:31.187593 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca033db-6b45-4f05-8abb-a92504f624aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:58:31.188106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca033db-6b45-4f05-8abb-a92504f624aa service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:31.188552 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca033db-6b45-4f05-8abb-a92504f624aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:31.188817 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca033db-6b45-4f05-8abb-a92504f624aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:31.190729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-bca033db-6b45-4f05-8abb-a92504f624aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:31.285431 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:31.345155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca033db-6b45-4f05-8abb-a92504f624aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:58:31.345155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca033db-6b45-4f05-8abb-a92504f624aa service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:58:31.369775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 688/1373] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:58:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 11:58:33.034220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874aa44-c9c7-49f4-aa7c-cf4001f53b17 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:33.034786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874aa44-c9c7-49f4-aa7c-cf4001f53b17 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:33.034786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874aa44-c9c7-49f4-aa7c-cf4001f53b17 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:33.036365 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 686/1374] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:33.042315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc1231-2399-4102-ae30-c4b1668b8c88 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:33.042756 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc1231-2399-4102-ae30-c4b1668b8c88 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:33.043002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc1231-2399-4102-ae30-c4b1668b8c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:33.125788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-41bc1231-2399-4102-ae30-c4b1668b8c88 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:33.127096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-41bc1231-2399-4102-ae30-c4b1668b8c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:33.232625 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 689/1375] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:58:33.550929 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874aa44-c9c7-49f4-aa7c-cf4001f53b17 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:58:33.552509 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9874aa44-c9c7-49f4-aa7c-cf4001f53b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:33.676540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:33.759018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874aa44-c9c7-49f4-aa7c-cf4001f53b17 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:58:33.952422 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946da85-bae8-476c-8cf4-308574195556 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:58:33.953695 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946da85-bae8-476c-8cf4-308574195556 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:33.953695 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946da85-bae8-476c-8cf4-308574195556 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:33.954075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946da85-bae8-476c-8cf4-308574195556 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:33.957019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d946da85-bae8-476c-8cf4-308574195556 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:34.034496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:34.106517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946da85-bae8-476c-8cf4-308574195556 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:58:34.109424 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946da85-bae8-476c-8cf4-308574195556 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:58:34.137432 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 687/1376] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:58:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:58:36.853338 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866bced-d2ce-4255-b994-2c85bf5fb8dc None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:36.854448 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866bced-d2ce-4255-b994-2c85bf5fb8dc None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:36.854448 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866bced-d2ce-4255-b994-2c85bf5fb8dc None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:36.860203 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 690/1377] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:58:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:36.860825 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95820916-091c-4960-a1ce-c7ae784e9efa None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:36.861249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95820916-091c-4960-a1ce-c7ae784e9efa None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:36.861429 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95820916-091c-4960-a1ce-c7ae784e9efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:36.937923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-95820916-091c-4960-a1ce-c7ae784e9efa None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:36.939155 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-95820916-091c-4960-a1ce-c7ae784e9efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:37.056663 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 688/1378] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:58:37.407185 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866bced-d2ce-4255-b994-2c85bf5fb8dc None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:58:37.410115 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9866bced-d2ce-4255-b994-2c85bf5fb8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:37.460583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:37.602548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866bced-d2ce-4255-b994-2c85bf5fb8dc None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:58:37.790839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:37.800037 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80d9db-7bf7-43c0-a653-7d6507564ea3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:58:37.801005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80d9db-7bf7-43c0-a653-7d6507564ea3 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:37.801644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80d9db-7bf7-43c0-a653-7d6507564ea3 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:37.805347 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80d9db-7bf7-43c0-a653-7d6507564ea3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:37.805347 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3c80d9db-7bf7-43c0-a653-7d6507564ea3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:37.982455 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80d9db-7bf7-43c0-a653-7d6507564ea3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:58:37.985379 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80d9db-7bf7-43c0-a653-7d6507564ea3 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:58:38.010821 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 691/1379] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:58:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:58:40.747059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca177afb-5540-4cc2-8a7c-16c96697b4eb None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:40.747786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca177afb-5540-4cc2-8a7c-16c96697b4eb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:40.747786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca177afb-5540-4cc2-8a7c-16c96697b4eb None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:40.748863 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 689/1380] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:40.754945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387ef92-5e4b-4f01-aee8-6672516a88a7 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:40.755575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387ef92-5e4b-4f01-aee8-6672516a88a7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:40.755575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387ef92-5e4b-4f01-aee8-6672516a88a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:40.838288 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a387ef92-5e4b-4f01-aee8-6672516a88a7 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:40.839968 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a387ef92-5e4b-4f01-aee8-6672516a88a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:40.932968 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:40.972631 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 692/1381] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 11:58:41.370787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca177afb-5540-4cc2-8a7c-16c96697b4eb None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:58:41.372493 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ca177afb-5540-4cc2-8a7c-16c96697b4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:41.411795 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:41.560601 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca177afb-5540-4cc2-8a7c-16c96697b4eb None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:58:41.690294 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:41.718103 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12a5b8-1bb4-47ca-b5a2-8bd53c96f4e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:58:41.718989 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12a5b8-1bb4-47ca-b5a2-8bd53c96f4e3 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:41.719348 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12a5b8-1bb4-47ca-b5a2-8bd53c96f4e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:41.719635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12a5b8-1bb4-47ca-b5a2-8bd53c96f4e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:41.721323 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6d12a5b8-1bb4-47ca-b5a2-8bd53c96f4e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:41.828101 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d12a5b8-1bb4-47ca-b5a2-8bd53c96f4e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:58:41.829722 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d12a5b8-1bb4-47ca-b5a2-8bd53c96f4e3 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:58:41.854883 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 690/1382] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:58:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:58:44.287561 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6817842-e16b-47e4-b3a9-ef44affe7615 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:44.288155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6817842-e16b-47e4-b3a9-ef44affe7615 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:44.288155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6817842-e16b-47e4-b3a9-ef44affe7615 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:44.289510 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 693/1383] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:44.296689 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40b202-2e39-485b-b0aa-5d068dd4c9e4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:44.297133 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40b202-2e39-485b-b0aa-5d068dd4c9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:44.297466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40b202-2e39-485b-b0aa-5d068dd4c9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:44.397825 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6d40b202-2e39-485b-b0aa-5d068dd4c9e4 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:44.399100 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6d40b202-2e39-485b-b0aa-5d068dd4c9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:44.462465 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:44.504501 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 691/1384] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:58:44.894792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6817842-e16b-47e4-b3a9-ef44affe7615 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:58:44.896477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d6817842-e16b-47e4-b3a9-ef44affe7615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:45.091575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6817842-e16b-47e4-b3a9-ef44affe7615 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:58:45.156730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:45.232605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85cfc4-71eb-4dcc-aec8-d5c146247391 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:58:45.233687 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85cfc4-71eb-4dcc-aec8-d5c146247391 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:45.234182 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85cfc4-71eb-4dcc-aec8-d5c146247391 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:45.234548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85cfc4-71eb-4dcc-aec8-d5c146247391 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:45.236793 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0b85cfc4-71eb-4dcc-aec8-d5c146247391 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:45.345975 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85cfc4-71eb-4dcc-aec8-d5c146247391 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:58:45.347566 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85cfc4-71eb-4dcc-aec8-d5c146247391 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:58:45.368355 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 694/1385] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:58:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:58:47.625325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a84490-1971-4e6f-9020-98f1af6ecfc7 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:47.625910 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a84490-1971-4e6f-9020-98f1af6ecfc7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:47.626017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a84490-1971-4e6f-9020-98f1af6ecfc7 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:47.627309 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 692/1386] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:47.633352 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb4509-78d5-425c-82d4-31e3aeb8fe7a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:47.633802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb4509-78d5-425c-82d4-31e3aeb8fe7a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:47.633945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb4509-78d5-425c-82d4-31e3aeb8fe7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:47.702044 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d9eb4509-78d5-425c-82d4-31e3aeb8fe7a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:47.703275 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d9eb4509-78d5-425c-82d4-31e3aeb8fe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:47.713344 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:47.825981 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 695/1387] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:58:48.111144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a84490-1971-4e6f-9020-98f1af6ecfc7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:58:48.112667 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a0a84490-1971-4e6f-9020-98f1af6ecfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:48.180639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307899ac-532b-4d5e-b58b-3c3462e940e3 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:48.180929 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307899ac-532b-4d5e-b58b-3c3462e940e3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:48.181213 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307899ac-532b-4d5e-b58b-3c3462e940e3 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:48.182620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 696/1388] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:58:48.187717 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85510db0-bcb7-40d1-8f7b-858a61de03c3 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:48.188005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85510db0-bcb7-40d1-8f7b-858a61de03c3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:48.188281 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85510db0-bcb7-40d1-8f7b-858a61de03c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:48.257342 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-85510db0-bcb7-40d1-8f7b-858a61de03c3 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:48.258873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-85510db0-bcb7-40d1-8f7b-858a61de03c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:48.268101 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a84490-1971-4e6f-9020-98f1af6ecfc7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:58:48.320257 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:48.354515 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:48.366335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 697/1389] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 11:58:48.409300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790da1c6-499b-4f53-be12-332cd4ac9466 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:58:48.409930 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790da1c6-499b-4f53-be12-332cd4ac9466 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:48.410214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790da1c6-499b-4f53-be12-332cd4ac9466 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:48.410485 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790da1c6-499b-4f53-be12-332cd4ac9466 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:48.412341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-790da1c6-499b-4f53-be12-332cd4ac9466 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:48.506331 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790da1c6-499b-4f53-be12-332cd4ac9466 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:58:48.507947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790da1c6-499b-4f53-be12-332cd4ac9466 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:58:48.527248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 693/1390] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:58:48.665728 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bf4eb-825d-40fd-8055-a5adcd153be9 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:48.666227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bf4eb-825d-40fd-8055-a5adcd153be9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:48.666331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bf4eb-825d-40fd-8055-a5adcd153be9 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:48.667705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 698/1391] 10.222.0.62 () {54 vars in 966 bytes} [Wed Sep 14 11:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:48.673369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddb16b-b240-45c2-8acc-662ae47e80bc None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:48.673810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddb16b-b240-45c2-8acc-662ae47e80bc None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:48.673947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddb16b-b240-45c2-8acc-662ae47e80bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:48.741948 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:48.769287 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2cddb16b-b240-45c2-8acc-662ae47e80bc None None] MFA Rules not processed for user `4b05d0e65e9645f3810a6d96c8fed9fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:48.771061 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2cddb16b-b240-45c2-8acc-662ae47e80bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:48.866060 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 694/1392] 10.222.0.62 () {58 vars in 1063 bytes} [Wed Sep 14 11:58:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 11:58:48.875426 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bf4eb-825d-40fd-8055-a5adcd153be9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:58:48.877118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7e2bf4eb-825d-40fd-8055-a5adcd153be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:48.999949 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bf4eb-825d-40fd-8055-a5adcd153be9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:58:49.080138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402374b-27d3-48da-b055-2f042345af67 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b05d0e65e9645f3810a6d96c8fed9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:58:49.080753 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402374b-27d3-48da-b055-2f042345af67 service glance] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:49.081033 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402374b-27d3-48da-b055-2f042345af67 service glance] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:49.081310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402374b-27d3-48da-b055-2f042345af67 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:49.083029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8402374b-27d3-48da-b055-2f042345af67 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:49.126917 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:49.196080 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402374b-27d3-48da-b055-2f042345af67 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:58:49.197517 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402374b-27d3-48da-b055-2f042345af67 service glance] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:58:49.217115 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 699/1393] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:58:50.140465 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f138425-6111-43d4-8605-ef047f542210 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:50.140984 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f138425-6111-43d4-8605-ef047f542210 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:50.141091 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f138425-6111-43d4-8605-ef047f542210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:50.211345 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6f138425-6111-43d4-8605-ef047f542210 None None] MFA Rules not processed for user `d76a164ed7e14d9eb2db016b68a84d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:50.212604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6f138425-6111-43d4-8605-ef047f542210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:50.302332 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 695/1394] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 14 11:58:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 11:58:50.349822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21141441-063b-43d6-a4f6-d98b9480b1ba None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:50.350399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21141441-063b-43d6-a4f6-d98b9480b1ba None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:50.350510 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21141441-063b-43d6-a4f6-d98b9480b1ba None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:50.351833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 700/1395] 10.222.0.62 () {54 vars in 943 bytes} [Wed Sep 14 11:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:50.358864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe95748-ca19-4b73-8d5a-46e07e4da033 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:50.359159 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe95748-ca19-4b73-8d5a-46e07e4da033 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:50.359442 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe95748-ca19-4b73-8d5a-46e07e4da033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:50.369549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:50.440419 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-bbe95748-ca19-4b73-8d5a-46e07e4da033 None None] MFA Rules not processed for user `d1047c7d18c1466d989cd895117cd807`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:50.441643 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-bbe95748-ca19-4b73-8d5a-46e07e4da033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:50.551258 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 696/1396] 10.222.0.62 () {58 vars in 1040 bytes} [Wed Sep 14 11:58:50 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:58:50.560792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21141441-063b-43d6-a4f6-d98b9480b1ba None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:58:50.562380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-21141441-063b-43d6-a4f6-d98b9480b1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:50.709002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21141441-063b-43d6-a4f6-d98b9480b1ba None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:58:50.723147 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:50.822161 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526cf39-d30a-4810-81c8-704427c7f5d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1047c7d18c1466d989cd895117cd807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:58:50.822835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526cf39-d30a-4810-81c8-704427c7f5d9 service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:50.823118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526cf39-d30a-4810-81c8-704427c7f5d9 service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:50.823390 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526cf39-d30a-4810-81c8-704427c7f5d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:50.825114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7526cf39-d30a-4810-81c8-704427c7f5d9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:50.896397 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96410f56-3a9b-4e71-8c97-e78d0168a74b None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:50.896783 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96410f56-3a9b-4e71-8c97-e78d0168a74b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:50.897110 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96410f56-3a9b-4e71-8c97-e78d0168a74b None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:50.902790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 697/1397] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 11:58:50.904202 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91469c-793c-4c91-b9ac-7a509e50a2a3 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:50.904492 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91469c-793c-4c91-b9ac-7a509e50a2a3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:50.904989 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91469c-793c-4c91-b9ac-7a509e50a2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:50.923054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526cf39-d30a-4810-81c8-704427c7f5d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:58:50.924656 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526cf39-d30a-4810-81c8-704427c7f5d9 service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:58:50.933765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 701/1398] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 14 11:58:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 11:58:50.988435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2b91469c-793c-4c91-b9ac-7a509e50a2a3 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:50.989661 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2b91469c-793c-4c91-b9ac-7a509e50a2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:51.009994 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:51.096666 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 698/1399] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 11:58:51.392722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526cf39-d30a-4810-81c8-704427c7f5d9 service swift] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:58:51.394700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7526cf39-d30a-4810-81c8-704427c7f5d9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:51.448824 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:51.505952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526cf39-d30a-4810-81c8-704427c7f5d9 service swift] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:58:51.642448 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105efd5-ce5f-419c-bfa7-f7617a85b128 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:58:51.642973 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105efd5-ce5f-419c-bfa7-f7617a85b128 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:51.643251 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105efd5-ce5f-419c-bfa7-f7617a85b128 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:51.643524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105efd5-ce5f-419c-bfa7-f7617a85b128 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:51.645137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a105efd5-ce5f-419c-bfa7-f7617a85b128 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:51.720547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:51.753805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105efd5-ce5f-419c-bfa7-f7617a85b128 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:58:51.755637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105efd5-ce5f-419c-bfa7-f7617a85b128 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:58:51.781060 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 702/1400] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:58:54.197247 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782a1f6-6ae3-415d-93a4-78df1c570f6a None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:54.197819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782a1f6-6ae3-415d-93a4-78df1c570f6a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:54.197819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782a1f6-6ae3-415d-93a4-78df1c570f6a None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:54.199368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 699/1401] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:54.208276 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f88b9b-b089-42d0-bab4-17180cdaea3b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:54.209014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f88b9b-b089-42d0-bab4-17180cdaea3b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:54.209014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f88b9b-b089-42d0-bab4-17180cdaea3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:54.295155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-23f88b9b-b089-42d0-bab4-17180cdaea3b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:54.342433 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-23f88b9b-b089-42d0-bab4-17180cdaea3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:54.464548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 703/1402] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:58:54.781502 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782a1f6-6ae3-415d-93a4-78df1c570f6a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:58:54.783093 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7782a1f6-6ae3-415d-93a4-78df1c570f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:54.967525 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:54.969511 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782a1f6-6ae3-415d-93a4-78df1c570f6a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:58:55.118121 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa3397-13fc-4c4c-93ea-3465a4f17aec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:58:55.119261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa3397-13fc-4c4c-93ea-3465a4f17aec service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:55.119792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa3397-13fc-4c4c-93ea-3465a4f17aec service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:55.120322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa3397-13fc-4c4c-93ea-3465a4f17aec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:55.122632 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-09fa3397-13fc-4c4c-93ea-3465a4f17aec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:55.222723 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa3397-13fc-4c4c-93ea-3465a4f17aec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:58:55.224452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa3397-13fc-4c4c-93ea-3465a4f17aec service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:58:55.243588 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 700/1403] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:58:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:58:57.783050 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f72259-9f3a-4f88-b7a4-cc075fc15196 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:57.784028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f72259-9f3a-4f88-b7a4-cc075fc15196 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:57.784541 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f72259-9f3a-4f88-b7a4-cc075fc15196 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:57.786934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 704/1404] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:58:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:58:57.793044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953477fd-8ce0-4395-9fab-2ca6d29ac699 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:57.796659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953477fd-8ce0-4395-9fab-2ca6d29ac699 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:57.796659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953477fd-8ce0-4395-9fab-2ca6d29ac699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:57.800074 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:57.888908 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-953477fd-8ce0-4395-9fab-2ca6d29ac699 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:58:57.890639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-953477fd-8ce0-4395-9fab-2ca6d29ac699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:58.013837 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 701/1405] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:58:58.318251 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f72259-9f3a-4f88-b7a4-cc075fc15196 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:58:58.319850 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f6f72259-9f3a-4f88-b7a4-cc075fc15196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:58.383053 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:58.527078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f72259-9f3a-4f88-b7a4-cc075fc15196 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:58:58.666588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c79c7f-fa9d-49f5-b7dd-88de37a0178e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:58:58.667215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c79c7f-fa9d-49f5-b7dd-88de37a0178e service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:58:58.667499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c79c7f-fa9d-49f5-b7dd-88de37a0178e service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:58:58.667772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c79c7f-fa9d-49f5-b7dd-88de37a0178e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:58:58.669489 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-72c79c7f-fa9d-49f5-b7dd-88de37a0178e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:58:58.672109 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:58:58.797190 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c79c7f-fa9d-49f5-b7dd-88de37a0178e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:58:58.798867 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c79c7f-fa9d-49f5-b7dd-88de37a0178e service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:58:58.829127 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 705/1406] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:58:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:59:01.245463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a86e3-3ce4-49f3-949a-446f66edd1ff None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:01.246632 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a86e3-3ce4-49f3-949a-446f66edd1ff None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:01.247005 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a86e3-3ce4-49f3-949a-446f66edd1ff None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:01.249907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 702/1407] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 11:59:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:01.255537 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ddadf-4387-4a5c-8638-5a3fa71339f6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:01.257711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ddadf-4387-4a5c-8638-5a3fa71339f6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:01.258362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ddadf-4387-4a5c-8638-5a3fa71339f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:01.346022 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9d4ddadf-4387-4a5c-8638-5a3fa71339f6 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:01.347834 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9d4ddadf-4387-4a5c-8638-5a3fa71339f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:01.432206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:01.474038 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 706/1408] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 11:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:59:01.771961 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a86e3-3ce4-49f3-949a-446f66edd1ff None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:59:01.773514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-585a86e3-3ce4-49f3-949a-446f66edd1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:01.897691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:01.985634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a86e3-3ce4-49f3-949a-446f66edd1ff None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:59:02.152326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad4ff9-6756-4fb5-b4fc-857e2e5eabb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:59:02.152946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad4ff9-6756-4fb5-b4fc-857e2e5eabb3 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:02.153219 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad4ff9-6756-4fb5-b4fc-857e2e5eabb3 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:02.153485 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad4ff9-6756-4fb5-b4fc-857e2e5eabb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:02.155252 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e6ad4ff9-6756-4fb5-b4fc-857e2e5eabb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:02.343526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:02.401354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad4ff9-6756-4fb5-b4fc-857e2e5eabb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:59:02.403405 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad4ff9-6756-4fb5-b4fc-857e2e5eabb3 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:59:02.428041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 703/1409] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:59:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:59:11.469911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=9 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:12.421995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=25 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:16.878409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ddbe9-2866-42fc-b5d7-6957491c8730 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:16.878991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ddbe9-2866-42fc-b5d7-6957491c8730 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:16.879107 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ddbe9-2866-42fc-b5d7-6957491c8730 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:16.881604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 707/1410] 10.222.0.62 () {54 vars in 918 bytes} [Wed Sep 14 11:59:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:16.887262 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cd539-9407-4ecb-a8c2-d8d190bd059b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:16.887786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cd539-9407-4ecb-a8c2-d8d190bd059b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:16.887786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cd539-9407-4ecb-a8c2-d8d190bd059b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:16.979857 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6a0cd539-9407-4ecb-a8c2-d8d190bd059b None None] MFA Rules not processed for user `0ca10dd7ed714edb8b87017deecc41b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:16.981093 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6a0cd539-9407-4ecb-a8c2-d8d190bd059b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:17.116209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 704/1411] 10.222.0.62 () {58 vars in 1015 bytes} [Wed Sep 14 11:59:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:59:17.131723 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13d8a2-0219-4852-a373-303c2f995ac3 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:17.132015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13d8a2-0219-4852-a373-303c2f995ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:17.132300 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13d8a2-0219-4852-a373-303c2f995ac3 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:17.133768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 708/1412] 10.222.0.62 () {54 vars in 918 bytes} [Wed Sep 14 11:59:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:17.139049 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae85d9-cdd2-47d8-b8d7-52aa7476880a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:17.139350 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae85d9-cdd2-47d8-b8d7-52aa7476880a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:17.139633 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae85d9-cdd2-47d8-b8d7-52aa7476880a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:17.217725 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:17.220628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1fae85d9-cdd2-47d8-b8d7-52aa7476880a None None] MFA Rules not processed for user `38b093bc4c1d4ebcbd2a6bb16b70c029`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:17.221891 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1fae85d9-cdd2-47d8-b8d7-52aa7476880a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:17.347178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 705/1413] 10.222.0.62 () {58 vars in 1015 bytes} [Wed Sep 14 11:59:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:59:19.971964 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458f0ed-7dcb-4e9c-bea0-8d77c9d30883 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:19.972568 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458f0ed-7dcb-4e9c-bea0-8d77c9d30883 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:19.972568 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458f0ed-7dcb-4e9c-bea0-8d77c9d30883 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:19.974119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 11:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:19.980121 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8147d8e-dbf7-417f-81dd-198abf6e454a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:19.980971 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8147d8e-dbf7-417f-81dd-198abf6e454a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:19.980971 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8147d8e-dbf7-417f-81dd-198abf6e454a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:20.061491 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e8147d8e-dbf7-417f-81dd-198abf6e454a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:20.062745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e8147d8e-dbf7-417f-81dd-198abf6e454a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:20.120515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:20.165286 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 11:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:59:20.454978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458f0ed-7dcb-4e9c-bea0-8d77c9d30883 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:59:20.456471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8458f0ed-7dcb-4e9c-bea0-8d77c9d30883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:20.627955 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458f0ed-7dcb-4e9c-bea0-8d77c9d30883 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:59:20.718635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:20.765488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cded358-950b-4aa3-a134-e7bf6395e804 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:59:20.766175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cded358-950b-4aa3-a134-e7bf6395e804 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:20.766497 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cded358-950b-4aa3-a134-e7bf6395e804 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:20.766838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cded358-950b-4aa3-a134-e7bf6395e804 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:20.768594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6cded358-950b-4aa3-a134-e7bf6395e804 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:20.874822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cded358-950b-4aa3-a134-e7bf6395e804 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:59:20.876614 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cded358-950b-4aa3-a134-e7bf6395e804 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:59:20.897955 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 710/1416] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:59:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:59:22.052155 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c3635-b257-464f-b29e-0b343b6e40b7 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:22.052600 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c3635-b257-464f-b29e-0b343b6e40b7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:22.052721 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c3635-b257-464f-b29e-0b343b6e40b7 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:22.054077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 11:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:22.060521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73af30-cb33-4ccf-b8cb-80999f4442e4 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:22.060989 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73af30-cb33-4ccf-b8cb-80999f4442e4 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:22.064824 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73af30-cb33-4ccf-b8cb-80999f4442e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:22.140870 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-6b73af30-cb33-4ccf-b8cb-80999f4442e4 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:22.142476 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6b73af30-cb33-4ccf-b8cb-80999f4442e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:22.172254 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:22.244273 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 11:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:59:22.393778 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c3635-b257-464f-b29e-0b343b6e40b7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:59:22.395232 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b61c3635-b257-464f-b29e-0b343b6e40b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:22.541611 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c3635-b257-464f-b29e-0b343b6e40b7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:59:22.583557 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:22.656635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e796eea-3e69-433d-aa64-9b5a9cd7e7a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:59:22.657159 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e796eea-3e69-433d-aa64-9b5a9cd7e7a2 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:22.657445 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e796eea-3e69-433d-aa64-9b5a9cd7e7a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:22.657718 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e796eea-3e69-433d-aa64-9b5a9cd7e7a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:22.659364 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5e796eea-3e69-433d-aa64-9b5a9cd7e7a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:22.750669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e796eea-3e69-433d-aa64-9b5a9cd7e7a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:59:22.752251 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e796eea-3e69-433d-aa64-9b5a9cd7e7a2 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:59:22.771075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 708/1419] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:59:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:59:23.927812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9dae27-51c1-443d-896d-19e4b8637867 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:23.928361 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9dae27-51c1-443d-896d-19e4b8637867 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:23.928361 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9dae27-51c1-443d-896d-19e4b8637867 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:23.929703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 11:59:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:23.936076 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c4e1c-43c5-4ebf-8d9e-6a6488a72595 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:23.936540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c4e1c-43c5-4ebf-8d9e-6a6488a72595 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:23.936662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c4e1c-43c5-4ebf-8d9e-6a6488a72595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:23.992005 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:24.015269 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9a2c4e1c-43c5-4ebf-8d9e-6a6488a72595 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:24.016478 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9a2c4e1c-43c5-4ebf-8d9e-6a6488a72595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:24.116776 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 11:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:59:24.267367 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9dae27-51c1-443d-896d-19e4b8637867 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:59:24.268902 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-bf9dae27-51c1-443d-896d-19e4b8637867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:24.420059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9dae27-51c1-443d-896d-19e4b8637867 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:59:24.432813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:24.534561 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8998d4ab-5b8d-4437-8ae1-5363426005b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:59:24.535494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998d4ab-5b8d-4437-8ae1-5363426005b3 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:24.535784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998d4ab-5b8d-4437-8ae1-5363426005b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:24.536061 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998d4ab-5b8d-4437-8ae1-5363426005b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:24.537673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8998d4ab-5b8d-4437-8ae1-5363426005b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:24.630571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8998d4ab-5b8d-4437-8ae1-5363426005b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:59:24.632186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8998d4ab-5b8d-4437-8ae1-5363426005b3 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:59:24.650625 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 713/1422] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:59:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:59:25.763744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e894b-660e-4347-9e75-146446a2f9dd None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:25.764551 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e894b-660e-4347-9e75-146446a2f9dd None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:25.764745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e894b-660e-4347-9e75-146446a2f9dd None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:25.766237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 11:59:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:25.773328 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a69fb1-cea2-4fb4-8164-817995927b7b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:25.773804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a69fb1-cea2-4fb4-8164-817995927b7b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:25.773950 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a69fb1-cea2-4fb4-8164-817995927b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:25.793906 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:25.852593 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e4a69fb1-cea2-4fb4-8164-817995927b7b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:25.853831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e4a69fb1-cea2-4fb4-8164-817995927b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:25.954289 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 11:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:59:26.103920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e894b-660e-4347-9e75-146446a2f9dd None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:59:26.105627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1d3e894b-660e-4347-9e75-146446a2f9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:26.220205 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:26.256033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e894b-660e-4347-9e75-146446a2f9dd None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:59:26.371823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d97db-8d75-4981-8006-950238689811 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:59:26.372439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d97db-8d75-4981-8006-950238689811 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:26.372716 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d97db-8d75-4981-8006-950238689811 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:26.372990 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d97db-8d75-4981-8006-950238689811 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:26.374642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b44d97db-8d75-4981-8006-950238689811 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:26.454089 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:26.468271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d97db-8d75-4981-8006-950238689811 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:59:26.469877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d97db-8d75-4981-8006-950238689811 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:59:26.488812 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 711/1425] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:59:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:59:27.629664 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc3a5f-2d3b-45de-be90-761ba53666e0 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:27.630784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc3a5f-2d3b-45de-be90-761ba53666e0 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:27.630784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc3a5f-2d3b-45de-be90-761ba53666e0 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:27.632198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 11:59:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:27.638727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71155c-97e6-42d2-b639-766fb4ae24c6 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:27.639178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71155c-97e6-42d2-b639-766fb4ae24c6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:27.639304 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71155c-97e6-42d2-b639-766fb4ae24c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:27.717430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9a71155c-97e6-42d2-b639-766fb4ae24c6 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:27.718682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9a71155c-97e6-42d2-b639-766fb4ae24c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:27.816535 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 11:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:59:27.965053 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc3a5f-2d3b-45de-be90-761ba53666e0 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:59:27.966684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-02fc3a5f-2d3b-45de-be90-761ba53666e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:28.056353 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:28.119380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc3a5f-2d3b-45de-be90-761ba53666e0 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:59:28.234332 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f2922-fc5e-433f-b113-8c1348200cba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:59:28.234952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f2922-fc5e-433f-b113-8c1348200cba service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:28.235231 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f2922-fc5e-433f-b113-8c1348200cba service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:28.235503 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f2922-fc5e-433f-b113-8c1348200cba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:28.237111 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-973f2922-fc5e-433f-b113-8c1348200cba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:28.288425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:28.335288 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f2922-fc5e-433f-b113-8c1348200cba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:59:28.336843 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f2922-fc5e-433f-b113-8c1348200cba service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:59:28.356792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 716/1428] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:59:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:59:29.524637 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300fb30-ce35-40d2-a993-7f728d1619e0 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:29.525347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300fb30-ce35-40d2-a993-7f728d1619e0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:29.525347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300fb30-ce35-40d2-a993-7f728d1619e0 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:29.526735 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 11:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:29.534244 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d3edb-3b21-4378-968f-b076bc194cfe None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:29.535193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d3edb-3b21-4378-968f-b076bc194cfe None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:29.535193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d3edb-3b21-4378-968f-b076bc194cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:29.621767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ad9d3edb-3b21-4378-968f-b076bc194cfe None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:29.623064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ad9d3edb-3b21-4378-968f-b076bc194cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:29.742666 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 11:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:59:29.895859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300fb30-ce35-40d2-a993-7f728d1619e0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:59:29.897437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8300fb30-ce35-40d2-a993-7f728d1619e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:29.953616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:30.055056 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300fb30-ce35-40d2-a993-7f728d1619e0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:59:30.173970 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd3658-a17d-4132-98c6-65e7b8fccb2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:59:30.174590 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd3658-a17d-4132-98c6-65e7b8fccb2f service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:30.174870 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd3658-a17d-4132-98c6-65e7b8fccb2f service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:30.175143 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd3658-a17d-4132-98c6-65e7b8fccb2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:30.176768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d2cd3658-a17d-4132-98c6-65e7b8fccb2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:30.184077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:30.268596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd3658-a17d-4132-98c6-65e7b8fccb2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:59:30.270283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd3658-a17d-4132-98c6-65e7b8fccb2f service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:59:30.289256 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 714/1431] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:59:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:59:31.421314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf8091-0c06-4589-a0f6-03ca1b6502eb None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:31.421920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf8091-0c06-4589-a0f6-03ca1b6502eb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:31.421920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf8091-0c06-4589-a0f6-03ca1b6502eb None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:31.423266 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 11:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:31.429576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13c92f-29c2-475d-8d64-c1c7abe1a26f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:31.430079 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13c92f-29c2-475d-8d64-c1c7abe1a26f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:31.430189 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13c92f-29c2-475d-8d64-c1c7abe1a26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:31.507124 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3c13c92f-29c2-475d-8d64-c1c7abe1a26f None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:31.508351 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3c13c92f-29c2-475d-8d64-c1c7abe1a26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:31.577162 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:31.606615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 11:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:59:31.759512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf8091-0c06-4589-a0f6-03ca1b6502eb None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:59:31.761272 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6fbf8091-0c06-4589-a0f6-03ca1b6502eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:31.779518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:31.913238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf8091-0c06-4589-a0f6-03ca1b6502eb None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:59:32.002016 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:32.027988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba382c3-98e9-4540-b92e-7e933f839620 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:59:32.028518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba382c3-98e9-4540-b92e-7e933f839620 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:32.028799 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba382c3-98e9-4540-b92e-7e933f839620 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:32.029072 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba382c3-98e9-4540-b92e-7e933f839620 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:32.030746 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0ba382c3-98e9-4540-b92e-7e933f839620 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:32.123567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba382c3-98e9-4540-b92e-7e933f839620 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:59:32.125149 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba382c3-98e9-4540-b92e-7e933f839620 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:59:32.143614 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 719/1434] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:59:33.270281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd4ed6-db86-4620-8c7d-9a29e660e64b None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:33.271372 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd4ed6-db86-4620-8c7d-9a29e660e64b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:33.271372 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd4ed6-db86-4620-8c7d-9a29e660e64b None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:33.272769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 11:59:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:33.279691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643828d-abba-4648-9c4b-60c925bca32a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:33.280235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643828d-abba-4648-9c4b-60c925bca32a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:33.280235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643828d-abba-4648-9c4b-60c925bca32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:33.360854 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c643828d-abba-4648-9c4b-60c925bca32a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:33.362117 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c643828d-abba-4648-9c4b-60c925bca32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:33.408807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:33.467889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 11:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:59:33.619056 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd4ed6-db86-4620-8c7d-9a29e660e64b None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:59:33.620567 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-06dd4ed6-db86-4620-8c7d-9a29e660e64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:33.786229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd4ed6-db86-4620-8c7d-9a29e660e64b None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:59:33.860959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:33.937980 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360eb04-196e-49ac-a56f-3c2b087c1f6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:59:33.938605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360eb04-196e-49ac-a56f-3c2b087c1f6b service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:33.938885 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360eb04-196e-49ac-a56f-3c2b087c1f6b service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:33.939155 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360eb04-196e-49ac-a56f-3c2b087c1f6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:33.940908 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9360eb04-196e-49ac-a56f-3c2b087c1f6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:34.041807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9360eb04-196e-49ac-a56f-3c2b087c1f6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:59:34.043447 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9360eb04-196e-49ac-a56f-3c2b087c1f6b service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:59:34.062187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 717/1437] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:59:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:59:35.196606 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f9a20-a0b5-4694-9b9e-5fa922d988e7 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:35.197209 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f9a20-a0b5-4694-9b9e-5fa922d988e7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:35.197209 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f9a20-a0b5-4694-9b9e-5fa922d988e7 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:35.198613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 11:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:35.205217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87035b1c-2bce-4913-937f-08036aef2302 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:35.205708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87035b1c-2bce-4913-937f-08036aef2302 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:35.205832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87035b1c-2bce-4913-937f-08036aef2302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:35.284482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-87035b1c-2bce-4913-937f-08036aef2302 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:35.286035 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-87035b1c-2bce-4913-937f-08036aef2302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:35.288224 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:35.389489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 11:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:59:35.540187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f9a20-a0b5-4694-9b9e-5fa922d988e7 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:59:35.541845 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fa4f9a20-a0b5-4694-9b9e-5fa922d988e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:35.694605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f9a20-a0b5-4694-9b9e-5fa922d988e7 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:59:35.724014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:35.841083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb017e4-390a-4457-95ac-4f0bafc60bd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:59:35.841083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb017e4-390a-4457-95ac-4f0bafc60bd4 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:35.841083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb017e4-390a-4457-95ac-4f0bafc60bd4 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:35.841083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb017e4-390a-4457-95ac-4f0bafc60bd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:35.841467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3fb017e4-390a-4457-95ac-4f0bafc60bd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:35.943346 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb017e4-390a-4457-95ac-4f0bafc60bd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:59:35.945075 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb017e4-390a-4457-95ac-4f0bafc60bd4 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:59:35.964362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 722/1440] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:59:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:59:38.091540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877873c-e181-4a9e-bc10-ed2e4a8de159 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:38.092108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877873c-e181-4a9e-bc10-ed2e4a8de159 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:38.092108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877873c-e181-4a9e-bc10-ed2e4a8de159 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:38.093520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 11:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:38.099108 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce6272-7ec2-4e83-b549-c6d63019d07a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:38.099590 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce6272-7ec2-4e83-b549-c6d63019d07a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:38.099753 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce6272-7ec2-4e83-b549-c6d63019d07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:38.142419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:38.169097 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-37ce6272-7ec2-4e83-b549-c6d63019d07a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:38.170373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-37ce6272-7ec2-4e83-b549-c6d63019d07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:38.272430 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 11:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:59:38.560966 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877873c-e181-4a9e-bc10-ed2e4a8de159 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:59:38.562541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4877873c-e181-4a9e-bc10-ed2e4a8de159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:38.670092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877873c-e181-4a9e-bc10-ed2e4a8de159 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:59:38.704954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:38.752497 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414dfcee-b79f-4b29-a434-e2dcfd95ffa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:59:38.753108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414dfcee-b79f-4b29-a434-e2dcfd95ffa0 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:38.753403 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414dfcee-b79f-4b29-a434-e2dcfd95ffa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:38.753671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414dfcee-b79f-4b29-a434-e2dcfd95ffa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:38.755333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-414dfcee-b79f-4b29-a434-e2dcfd95ffa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:38.869390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414dfcee-b79f-4b29-a434-e2dcfd95ffa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:59:38.870915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414dfcee-b79f-4b29-a434-e2dcfd95ffa0 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:59:38.890558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 720/1443] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:59:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:59:41.203113 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d11cd-2284-4cac-8e15-33937dcbb69a None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:41.203661 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d11cd-2284-4cac-8e15-33937dcbb69a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:41.203661 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d11cd-2284-4cac-8e15-33937dcbb69a None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:41.205058 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 11:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:41.210911 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9411528-c51a-463f-8a68-64d56ecaaa7a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:41.211362 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9411528-c51a-463f-8a68-64d56ecaaa7a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:41.211469 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9411528-c51a-463f-8a68-64d56ecaaa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:41.277241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e9411528-c51a-463f-8a68-64d56ecaaa7a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:41.278466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e9411528-c51a-463f-8a68-64d56ecaaa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:41.292020 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:41.378973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 11:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:59:41.661556 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d11cd-2284-4cac-8e15-33937dcbb69a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:59:41.663123 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-669d11cd-2284-4cac-8e15-33937dcbb69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:41.772785 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d11cd-2284-4cac-8e15-33937dcbb69a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:59:41.785218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:41.847889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc81295-2dcd-418b-9531-a891f6dec5dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:59:41.848501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc81295-2dcd-418b-9531-a891f6dec5dd service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:41.848782 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc81295-2dcd-418b-9531-a891f6dec5dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:41.849052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc81295-2dcd-418b-9531-a891f6dec5dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:41.850673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3fc81295-2dcd-418b-9531-a891f6dec5dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:41.964294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc81295-2dcd-418b-9531-a891f6dec5dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:59:41.965730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc81295-2dcd-418b-9531-a891f6dec5dd service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:59:41.984344 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 725/1446] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:59:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:59:44.561691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace49591-3999-478b-b7a0-3c6f7bc431e6 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:44.562300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace49591-3999-478b-b7a0-3c6f7bc431e6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:44.562300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace49591-3999-478b-b7a0-3c6f7bc431e6 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:44.563640 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 11:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:44.569412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3e101-5778-4045-b097-66b9a8049d6e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:44.569825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3e101-5778-4045-b097-66b9a8049d6e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:44.569996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3e101-5778-4045-b097-66b9a8049d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:44.618057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:44.636676 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-eba3e101-5778-4045-b097-66b9a8049d6e None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:44.645297 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-eba3e101-5778-4045-b097-66b9a8049d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:44.737672 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:59:45.027755 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace49591-3999-478b-b7a0-3c6f7bc431e6 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:59:45.029358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ace49591-3999-478b-b7a0-3c6f7bc431e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:45.133278 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace49591-3999-478b-b7a0-3c6f7bc431e6 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:59:45.176250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:45.211899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066b149-4252-42a7-afb0-d46567b9cdc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:59:45.212503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066b149-4252-42a7-afb0-d46567b9cdc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:45.212781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066b149-4252-42a7-afb0-d46567b9cdc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:45.213053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066b149-4252-42a7-afb0-d46567b9cdc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:45.214678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5066b149-4252-42a7-afb0-d46567b9cdc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:45.331247 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066b149-4252-42a7-afb0-d46567b9cdc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:59:45.333333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066b149-4252-42a7-afb0-d46567b9cdc2 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:59:45.352013 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 723/1449] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:59:47.646520 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2667c72-c4e6-4b43-81cd-d645329fb325 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:47.647107 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2667c72-c4e6-4b43-81cd-d645329fb325 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:47.647107 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2667c72-c4e6-4b43-81cd-d645329fb325 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:47.648455 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 11:59:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:47.654469 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c4137-20ee-46e0-8aec-d92c0334cb02 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:47.656686 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c4137-20ee-46e0-8aec-d92c0334cb02 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:47.656686 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c4137-20ee-46e0-8aec-d92c0334cb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:47.726522 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ce9c4137-20ee-46e0-8aec-d92c0334cb02 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:47.727744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ce9c4137-20ee-46e0-8aec-d92c0334cb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:47.751925 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:47.842298 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 11:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:59:48.131983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2667c72-c4e6-4b43-81cd-d645329fb325 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:59:48.133549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b2667c72-c4e6-4b43-81cd-d645329fb325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:48.245315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2667c72-c4e6-4b43-81cd-d645329fb325 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:59:48.267530 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:48.324957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec77348-7cfb-4b75-9e9f-a0fcd02be836 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:59:48.325584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec77348-7cfb-4b75-9e9f-a0fcd02be836 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:48.325889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec77348-7cfb-4b75-9e9f-a0fcd02be836 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:48.326183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec77348-7cfb-4b75-9e9f-a0fcd02be836 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:48.327886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0ec77348-7cfb-4b75-9e9f-a0fcd02be836 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:48.442860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec77348-7cfb-4b75-9e9f-a0fcd02be836 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:59:48.444213 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec77348-7cfb-4b75-9e9f-a0fcd02be836 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:59:48.463243 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 728/1452] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:59:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:59:50.742396 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bbe5e-6f56-4a3a-995a-3471707115f9 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:50.742998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bbe5e-6f56-4a3a-995a-3471707115f9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:50.743095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bbe5e-6f56-4a3a-995a-3471707115f9 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:50.744457 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 11:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:50.750368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f769c2-a9c1-4654-beaf-c9a1ea7a7eb5 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:50.750861 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f769c2-a9c1-4654-beaf-c9a1ea7a7eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:50.750957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f769c2-a9c1-4654-beaf-c9a1ea7a7eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:50.818558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:50.821966 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-68f769c2-a9c1-4654-beaf-c9a1ea7a7eb5 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:50.823230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-68f769c2-a9c1-4654-beaf-c9a1ea7a7eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:50.923539 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 11:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 11:59:51.210227 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735bbe5e-6f56-4a3a-995a-3471707115f9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:59:51.211787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-735bbe5e-6f56-4a3a-995a-3471707115f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:51.323557 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735bbe5e-6f56-4a3a-995a-3471707115f9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:59:51.374792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:51.404306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea7686-68a4-4d18-88b6-a16d4c744f5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:59:51.404920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea7686-68a4-4d18-88b6-a16d4c744f5f service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:51.405206 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea7686-68a4-4d18-88b6-a16d4c744f5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:51.405486 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea7686-68a4-4d18-88b6-a16d4c744f5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:51.407156 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fbea7686-68a4-4d18-88b6-a16d4c744f5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:51.519285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea7686-68a4-4d18-88b6-a16d4c744f5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:59:51.520658 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea7686-68a4-4d18-88b6-a16d4c744f5f service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:59:51.539537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 726/1455] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:59:53.768203 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a8f9e-c368-425d-a5ea-9f4fa4a4420e None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:53.768755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a8f9e-c368-425d-a5ea-9f4fa4a4420e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:53.768755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a8f9e-c368-425d-a5ea-9f4fa4a4420e None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:53.770666 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 11:59:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:53.777158 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea4a0f-0800-4ef3-b45d-34d72d315388 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:53.777667 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea4a0f-0800-4ef3-b45d-34d72d315388 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:53.777783 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea4a0f-0800-4ef3-b45d-34d72d315388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:53.852751 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8bea4a0f-0800-4ef3-b45d-34d72d315388 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:53.854051 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8bea4a0f-0800-4ef3-b45d-34d72d315388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:53.888740 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:53.969099 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 11:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 11:59:54.255624 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a8f9e-c368-425d-a5ea-9f4fa4a4420e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:59:54.257959 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-578a8f9e-c368-425d-a5ea-9f4fa4a4420e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:54.385175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a8f9e-c368-425d-a5ea-9f4fa4a4420e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:59:54.419108 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:54.472314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e6f19-7933-4228-9f8d-1fc95fb3d47c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:59:54.472953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e6f19-7933-4228-9f8d-1fc95fb3d47c service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:54.473238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e6f19-7933-4228-9f8d-1fc95fb3d47c service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:54.473518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e6f19-7933-4228-9f8d-1fc95fb3d47c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:54.475187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-010e6f19-7933-4228-9f8d-1fc95fb3d47c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:54.602548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e6f19-7933-4228-9f8d-1fc95fb3d47c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:59:54.604052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e6f19-7933-4228-9f8d-1fc95fb3d47c service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:59:54.625840 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 731/1458] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:59:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 11:59:56.929585 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ecf446-a3bb-49b7-8ab9-5e27181bf46c None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:56.930291 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ecf446-a3bb-49b7-8ab9-5e27181bf46c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:56.930496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ecf446-a3bb-49b7-8ab9-5e27181bf46c None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:56.932712 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 11:59:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 11:59:56.944785 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ced0da-5d79-48e2-9e02-bb1c0a971d5b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:56.945691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ced0da-5d79-48e2-9e02-bb1c0a971d5b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:56.945691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ced0da-5d79-48e2-9e02-bb1c0a971d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:57.048026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-76ced0da-5d79-48e2-9e02-bb1c0a971d5b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 11:59:57.050330 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-76ced0da-5d79-48e2-9e02-bb1c0a971d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:57.069525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:57.184821 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 11:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 11:59:57.487333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ecf446-a3bb-49b7-8ab9-5e27181bf46c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 11:59:57.489445 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-18ecf446-a3bb-49b7-8ab9-5e27181bf46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:57.619638 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ecf446-a3bb-49b7-8ab9-5e27181bf46c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 11:59:57.702120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 11:59:57.719975 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ecf6a6-ed7b-4c6a-9864-d20f851cc118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 11:59:57.721453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ecf6a6-ed7b-4c6a-9864-d20f851cc118 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 11:59:57.721453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ecf6a6-ed7b-4c6a-9864-d20f851cc118 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 11:59:57.721453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ecf6a6-ed7b-4c6a-9864-d20f851cc118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 11:59:57.722937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-21ecf6a6-ed7b-4c6a-9864-d20f851cc118 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 11:59:57.856326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ecf6a6-ed7b-4c6a-9864-d20f851cc118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 11:59:57.857852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ecf6a6-ed7b-4c6a-9864-d20f851cc118 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 11:59:57.878267 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 729/1461] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 11:59:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:00:00.147389 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f047b6-7638-41f3-a762-2e8e5869b3fe None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:00.148064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f047b6-7638-41f3-a762-2e8e5869b3fe None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:00.148064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f047b6-7638-41f3-a762-2e8e5869b3fe None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:00.149275 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 12:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:00:00.155034 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a018ca-0e2f-435d-923f-7c323692e1bb None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:00.155596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a018ca-0e2f-435d-923f-7c323692e1bb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:00.155596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a018ca-0e2f-435d-923f-7c323692e1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:00.227126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-68a018ca-0e2f-435d-923f-7c323692e1bb None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:00:00.228453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-68a018ca-0e2f-435d-923f-7c323692e1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:00.270696 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:00.334293 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 12:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:00:00.620863 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f047b6-7638-41f3-a762-2e8e5869b3fe None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:00:00.622444 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b3f047b6-7638-41f3-a762-2e8e5869b3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:00.754431 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f047b6-7638-41f3-a762-2e8e5869b3fe None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:00:00.795460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:00.835045 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c60f02-ae4a-4812-a907-1058342ce4fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:00:00.835706 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c60f02-ae4a-4812-a907-1058342ce4fd service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:00.835966 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c60f02-ae4a-4812-a907-1058342ce4fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:00.836457 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c60f02-ae4a-4812-a907-1058342ce4fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:00.838082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-59c60f02-ae4a-4812-a907-1058342ce4fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:00.959896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c60f02-ae4a-4812-a907-1058342ce4fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:00:00.961229 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c60f02-ae4a-4812-a907-1058342ce4fd service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:00:00.981985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 734/1464] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:00:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:00:03.255087 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d8b95-6d5f-4930-90f5-728bdee19b05 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:03.255671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d8b95-6d5f-4930-90f5-728bdee19b05 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:03.256095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d8b95-6d5f-4930-90f5-728bdee19b05 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:03.259929 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 12:00:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:00:03.268511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d5b56-94f6-4c6f-8604-7da90acecb54 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:03.269100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d5b56-94f6-4c6f-8604-7da90acecb54 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:03.270017 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d5b56-94f6-4c6f-8604-7da90acecb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:03.339375 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ff5d5b56-94f6-4c6f-8604-7da90acecb54 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:00:03.340621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ff5d5b56-94f6-4c6f-8604-7da90acecb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:03.359240 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:03.438402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 12:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:00:03.725215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d8b95-6d5f-4930-90f5-728bdee19b05 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:00:03.726812 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e04d8b95-6d5f-4930-90f5-728bdee19b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:03.865137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d8b95-6d5f-4930-90f5-728bdee19b05 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:00:03.955282 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:03.966497 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de4cec-a2d5-491b-8548-03272d35d13c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:00:03.967478 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de4cec-a2d5-491b-8548-03272d35d13c service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:03.967956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de4cec-a2d5-491b-8548-03272d35d13c service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:03.968435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de4cec-a2d5-491b-8548-03272d35d13c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:03.971213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-43de4cec-a2d5-491b-8548-03272d35d13c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:04.113876 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de4cec-a2d5-491b-8548-03272d35d13c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:00:04.114859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de4cec-a2d5-491b-8548-03272d35d13c service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:00:04.134131 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 732/1467] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:00:06.414902 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dad9da-f195-4152-9fa3-a05b2560db2b None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:06.415562 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dad9da-f195-4152-9fa3-a05b2560db2b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:06.415562 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dad9da-f195-4152-9fa3-a05b2560db2b None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:06.416832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 12:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:00:06.423973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4362973-56b8-41ef-be94-3cd2a1925cf3 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:06.424523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4362973-56b8-41ef-be94-3cd2a1925cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:06.424984 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4362973-56b8-41ef-be94-3cd2a1925cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:06.506982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a4362973-56b8-41ef-be94-3cd2a1925cf3 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:00:06.508242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a4362973-56b8-41ef-be94-3cd2a1925cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:06.568152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:06.623583 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 12:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:00:06.922729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dad9da-f195-4152-9fa3-a05b2560db2b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:00:06.924315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b4dad9da-f195-4152-9fa3-a05b2560db2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:07.042641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dad9da-f195-4152-9fa3-a05b2560db2b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:00:07.096498 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:07.129756 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822ba4a-1df1-4edc-805f-22dc2085de98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:00:07.131139 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822ba4a-1df1-4edc-805f-22dc2085de98 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:07.131659 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822ba4a-1df1-4edc-805f-22dc2085de98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:07.132169 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822ba4a-1df1-4edc-805f-22dc2085de98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:07.135220 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d822ba4a-1df1-4edc-805f-22dc2085de98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:07.270474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822ba4a-1df1-4edc-805f-22dc2085de98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:00:07.272089 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822ba4a-1df1-4edc-805f-22dc2085de98 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:00:07.292597 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 737/1470] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:00:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:00:16.649723 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=22 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:17.290033 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=66 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:27.733615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3fe42-72da-4266-a22e-ff2dec58e347 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:27.734285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3fe42-72da-4266-a22e-ff2dec58e347 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:27.734285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3fe42-72da-4266-a22e-ff2dec58e347 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:27.735582 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 734/1471] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 12:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:00:27.741516 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab218df1-c76c-4393-81ef-e0314eaab032 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:27.742411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab218df1-c76c-4393-81ef-e0314eaab032 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:27.742754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab218df1-c76c-4393-81ef-e0314eaab032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:27.817816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ab218df1-c76c-4393-81ef-e0314eaab032 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:00:27.819164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ab218df1-c76c-4393-81ef-e0314eaab032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:27.919316 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 738/1472] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 12:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:00:28.218924 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86cc23-7a26-4f4d-b9d1-a36e59f24a0f None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:28.219207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86cc23-7a26-4f4d-b9d1-a36e59f24a0f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:28.219482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86cc23-7a26-4f4d-b9d1-a36e59f24a0f None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:28.220833 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 735/1473] 10.222.0.62 () {54 vars in 966 bytes} [Wed Sep 14 12:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 12:00:28.226386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ea9a9-4fd5-4b40-9df1-6e1beae9f36d None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:28.226674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ea9a9-4fd5-4b40-9df1-6e1beae9f36d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:28.226957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ea9a9-4fd5-4b40-9df1-6e1beae9f36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:28.310691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-950ea9a9-4fd5-4b40-9df1-6e1beae9f36d None None] MFA Rules not processed for user `4b05d0e65e9645f3810a6d96c8fed9fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:00:28.311932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-950ea9a9-4fd5-4b40-9df1-6e1beae9f36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:28.334721 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:28.396789 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 739/1474] 10.222.0.62 () {58 vars in 1063 bytes} [Wed Sep 14 12:00:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:00:28.405067 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86cc23-7a26-4f4d-b9d1-a36e59f24a0f None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:00:28.406707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-da86cc23-7a26-4f4d-b9d1-a36e59f24a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:28.526133 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86cc23-7a26-4f4d-b9d1-a36e59f24a0f None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:00:28.604381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5921b0-0cd2-49ee-afdf-d5ed42d36d08 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b05d0e65e9645f3810a6d96c8fed9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:00:28.604989 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5921b0-0cd2-49ee-afdf-d5ed42d36d08 service glance] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:28.605270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5921b0-0cd2-49ee-afdf-d5ed42d36d08 service glance] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:28.605541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5921b0-0cd2-49ee-afdf-d5ed42d36d08 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:28.607178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5b5921b0-0cd2-49ee-afdf-d5ed42d36d08 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:28.662471 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:28.730654 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5921b0-0cd2-49ee-afdf-d5ed42d36d08 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:00:28.732102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5921b0-0cd2-49ee-afdf-d5ed42d36d08 service glance] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:00:28.754709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 736/1475] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:00:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:00:31.025362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08d324-6a3c-4964-9d2f-60254f991c23 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:31.025954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08d324-6a3c-4964-9d2f-60254f991c23 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:31.025954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08d324-6a3c-4964-9d2f-60254f991c23 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:31.027323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 740/1476] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 12:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:00:31.032724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3166f-c8f8-463d-bac8-87aad96d92ab None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:31.033180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3166f-c8f8-463d-bac8-87aad96d92ab None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:31.033286 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3166f-c8f8-463d-bac8-87aad96d92ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:31.109919 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9ae3166f-c8f8-463d-bac8-87aad96d92ab None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:00:31.111141 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9ae3166f-c8f8-463d-bac8-87aad96d92ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:31.214077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 737/1477] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 12:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:00:31.509348 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08d324-6a3c-4964-9d2f-60254f991c23 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:00:31.510599 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ea08d324-6a3c-4964-9d2f-60254f991c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:31.645477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08d324-6a3c-4964-9d2f-60254f991c23 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:00:31.701635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:31.746592 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a361df-0b62-4ed6-b759-bebea2e9c0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:00:31.747207 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a361df-0b62-4ed6-b759-bebea2e9c0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:31.747497 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a361df-0b62-4ed6-b759-bebea2e9c0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:31.747775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a361df-0b62-4ed6-b759-bebea2e9c0c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:31.748422 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a361df-0b62-4ed6-b759-bebea2e9c0c1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:00:31.750431 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-05a361df-0b62-4ed6-b759-bebea2e9c0c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 12:00:31.750431 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Sep 14 12:00:31.750431 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: Sep 14 12:00:31.751599 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a361df-0b62-4ed6-b759-bebea2e9c0c1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:00:31.771485 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 741/1478] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 14 12:00:31 2022] GET /identity/v3/projects => generated 2553 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:00:33.977373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5c520-4dbb-4288-8cc2-736283e97588 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:33.977972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5c520-4dbb-4288-8cc2-736283e97588 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:33.977972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5c520-4dbb-4288-8cc2-736283e97588 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:33.979479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 738/1479] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 12:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:00:33.985240 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331fad3-bc32-4163-beb4-99e51d910054 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:33.985809 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331fad3-bc32-4163-beb4-99e51d910054 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:33.986144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331fad3-bc32-4163-beb4-99e51d910054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:34.057777 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8331fad3-bc32-4163-beb4-99e51d910054 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:00:34.059050 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8331fad3-bc32-4163-beb4-99e51d910054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:34.175171 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 742/1480] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 12:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:00:34.477335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5c520-4dbb-4288-8cc2-736283e97588 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:00:34.478979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-53e5c520-4dbb-4288-8cc2-736283e97588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:34.490019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:34.634961 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5c520-4dbb-4288-8cc2-736283e97588 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:00:34.721349 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:34.755102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cd0c1-e222-419e-b30e-cc9082ccf001 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:00:34.755731 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cd0c1-e222-419e-b30e-cc9082ccf001 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:34.756044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cd0c1-e222-419e-b30e-cc9082ccf001 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:34.756342 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cd0c1-e222-419e-b30e-cc9082ccf001 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:34.758042 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ef2cd0c1-e222-419e-b30e-cc9082ccf001 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:34.857154 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cd0c1-e222-419e-b30e-cc9082ccf001 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:00:34.858788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cd0c1-e222-419e-b30e-cc9082ccf001 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:00:34.877906 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 739/1481] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:00:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:00:37.238345 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7f12b-2c08-461f-bb08-b76e8ab0a458 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:37.239282 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7f12b-2c08-461f-bb08-b76e8ab0a458 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:37.239282 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7f12b-2c08-461f-bb08-b76e8ab0a458 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:37.240918 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 743/1482] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 12:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:00:37.246639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8b300-cbd7-43db-9598-7dadfdc6979b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:37.247121 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8b300-cbd7-43db-9598-7dadfdc6979b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:37.247242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8b300-cbd7-43db-9598-7dadfdc6979b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:37.323734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-bab8b300-cbd7-43db-9598-7dadfdc6979b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:00:37.325090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-bab8b300-cbd7-43db-9598-7dadfdc6979b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:37.375106 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:37.433873 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 740/1483] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 12:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:00:37.726817 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7f12b-2c08-461f-bb08-b76e8ab0a458 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:00:37.728373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-71c7f12b-2c08-461f-bb08-b76e8ab0a458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:37.760884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:37.875996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7f12b-2c08-461f-bb08-b76e8ab0a458 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:00:37.955785 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333814a-8c79-426c-9630-584a074bdf29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:00:37.956398 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333814a-8c79-426c-9630-584a074bdf29 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:37.956677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333814a-8c79-426c-9630-584a074bdf29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:37.956948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333814a-8c79-426c-9630-584a074bdf29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:37.958616 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e333814a-8c79-426c-9630-584a074bdf29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:38.043412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:38.080881 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333814a-8c79-426c-9630-584a074bdf29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:00:38.082302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333814a-8c79-426c-9630-584a074bdf29 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:00:38.102067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 744/1484] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:00:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:00:40.390773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dec83e-e363-4f16-9785-77b3575305c0 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:40.391359 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dec83e-e363-4f16-9785-77b3575305c0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:40.391359 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dec83e-e363-4f16-9785-77b3575305c0 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:40.393264 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 741/1485] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 12:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:00:40.399373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312298a-68fe-4b46-a8b8-4a97e26d08db None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:40.399989 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312298a-68fe-4b46-a8b8-4a97e26d08db None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:40.399989 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312298a-68fe-4b46-a8b8-4a97e26d08db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:40.479001 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b312298a-68fe-4b46-a8b8-4a97e26d08db None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:00:40.480295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b312298a-68fe-4b46-a8b8-4a97e26d08db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:40.588996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 745/1486] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 12:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:00:40.891243 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dec83e-e363-4f16-9785-77b3575305c0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:00:40.893505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-01dec83e-e363-4f16-9785-77b3575305c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:41.022870 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dec83e-e363-4f16-9785-77b3575305c0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:00:41.098963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:41.100520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed34186-e60e-4191-92dc-d8dd8df04d53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:00:41.101172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed34186-e60e-4191-92dc-d8dd8df04d53 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:41.101458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed34186-e60e-4191-92dc-d8dd8df04d53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:41.101736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed34186-e60e-4191-92dc-d8dd8df04d53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:41.103390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0ed34186-e60e-4191-92dc-d8dd8df04d53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:41.217229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed34186-e60e-4191-92dc-d8dd8df04d53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:00:41.218812 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed34186-e60e-4191-92dc-d8dd8df04d53 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:00:41.237032 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 742/1487] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:00:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:00:43.590074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a00339-8bd8-42dd-b42d-61438fe6c6cf None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:43.590576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a00339-8bd8-42dd-b42d-61438fe6c6cf None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:43.590688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a00339-8bd8-42dd-b42d-61438fe6c6cf None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:43.591993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 746/1488] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 12:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:00:43.597222 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f2639-82ee-468e-baa8-44bbbcb713ef None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:43.597660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f2639-82ee-468e-baa8-44bbbcb713ef None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:43.597761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f2639-82ee-468e-baa8-44bbbcb713ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:43.666956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-591f2639-82ee-468e-baa8-44bbbcb713ef None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:00:43.668181 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-591f2639-82ee-468e-baa8-44bbbcb713ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:43.723522 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:43.768424 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 743/1489] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 12:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:00:44.058580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a00339-8bd8-42dd-b42d-61438fe6c6cf None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:00:44.060123 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-84a00339-8bd8-42dd-b42d-61438fe6c6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:44.096973 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:44.167003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a00339-8bd8-42dd-b42d-61438fe6c6cf None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:00:44.242780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d7755-2df5-4e28-a2da-51f76416b1f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:00:44.243397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d7755-2df5-4e28-a2da-51f76416b1f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:44.243679 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d7755-2df5-4e28-a2da-51f76416b1f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:44.243955 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d7755-2df5-4e28-a2da-51f76416b1f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:44.245583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ac6d7755-2df5-4e28-a2da-51f76416b1f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:44.330636 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:44.360129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d7755-2df5-4e28-a2da-51f76416b1f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:00:44.362227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d7755-2df5-4e28-a2da-51f76416b1f3 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:00:44.380599 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 747/1490] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:00:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:00:46.959334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486ebc6-e06b-43db-a635-28771708fc8a None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:46.959898 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486ebc6-e06b-43db-a635-28771708fc8a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:46.960007 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486ebc6-e06b-43db-a635-28771708fc8a None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:46.961479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 744/1491] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 14 12:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:00:46.967194 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11446f0-0799-4917-b026-782cce8a047b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:46.967654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11446f0-0799-4917-b026-782cce8a047b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:46.967803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11446f0-0799-4917-b026-782cce8a047b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:47.035324 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f11446f0-0799-4917-b026-782cce8a047b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:00:47.036562 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f11446f0-0799-4917-b026-782cce8a047b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:47.131872 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 748/1492] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 14 12:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:00:47.423470 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486ebc6-e06b-43db-a635-28771708fc8a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:00:47.425039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5486ebc6-e06b-43db-a635-28771708fc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:47.538111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486ebc6-e06b-43db-a635-28771708fc8a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:00:47.614966 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b31b3d-5384-4a75-872b-7607f9bc89f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:00:47.615570 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b31b3d-5384-4a75-872b-7607f9bc89f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:00:47.615847 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b31b3d-5384-4a75-872b-7607f9bc89f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:00:47.616120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b31b3d-5384-4a75-872b-7607f9bc89f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:00:47.617710 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-36b31b3d-5384-4a75-872b-7607f9bc89f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:00:47.626328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:47.741356 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b31b3d-5384-4a75-872b-7607f9bc89f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:00:47.743060 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b31b3d-5384-4a75-872b-7607f9bc89f9 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:00:47.763441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 745/1493] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:00:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:00:57.130340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=82 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:00:57.760971 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=50 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:02:17.657131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2362421-8149-490b-b767-1f6b4c8f3052 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:02:17.657641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2362421-8149-490b-b767-1f6b4c8f3052 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:02:17.657749 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2362421-8149-490b-b767-1f6b4c8f3052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:02:17.740075 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c2362421-8149-490b-b767-1f6b4c8f3052 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:02:17.740699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c2362421-8149-490b-b767-1f6b4c8f3052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:02:17.847307 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 749/1494] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:02:17.855734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e717c-0037-4e8a-bcbe-ee7f8d180c1d None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:02:17.856241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e717c-0037-4e8a-bcbe-ee7f8d180c1d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:02:17.856356 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e717c-0037-4e8a-bcbe-ee7f8d180c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:02:17.878661 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9f8e717c-0037-4e8a-bcbe-ee7f8d180c1d None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:02:17.880309 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9f8e717c-0037-4e8a-bcbe-ee7f8d180c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:02:17.889084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 746/1495] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:02:17.893954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2362421-8149-490b-b767-1f6b4c8f3052 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:02:17.904026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2362421-8149-490b-b767-1f6b4c8f3052 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:02:17.910055 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed11bf-7d32-43d2-a2c6-744c84649b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:02:17.910700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed11bf-7d32-43d2-a2c6-744c84649b1d admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:02:17.910983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed11bf-7d32-43d2-a2c6-744c84649b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:02:17.911265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed11bf-7d32-43d2-a2c6-744c84649b1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:02:17.911928 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed11bf-7d32-43d2-a2c6-744c84649b1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:02:17.912515 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed11bf-7d32-43d2-a2c6-744c84649b1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:02:17.913344 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-48ed11bf-7d32-43d2-a2c6-744c84649b1d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 14 12:02:17.913344 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Sep 14 12:02:17.913344 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: Sep 14 12:02:17.914312 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed11bf-7d32-43d2-a2c6-744c84649b1d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:02:17.926478 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 750/1496] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 12:02:17.935592 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e717c-0037-4e8a-bcbe-ee7f8d180c1d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:02:17.945079 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e717c-0037-4e8a-bcbe-ee7f8d180c1d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:02:17.949838 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d304f50-4c5b-4e81-bc3c-5d77b3e8eb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:02:17.950491 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d304f50-4c5b-4e81-bc3c-5d77b3e8eb0a admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:02:17.950827 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d304f50-4c5b-4e81-bc3c-5d77b3e8eb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:02:17.951630 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d304f50-4c5b-4e81-bc3c-5d77b3e8eb0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:02:17.952283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d304f50-4c5b-4e81-bc3c-5d77b3e8eb0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:02:17.954107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-5d304f50-4c5b-4e81-bc3c-5d77b3e8eb0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 12:02:17.954107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Sep 14 12:02:17.954107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: Sep 14 12:02:17.955295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d304f50-4c5b-4e81-bc3c-5d77b3e8eb0a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:02:18.037403 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 747/1497] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:02:17 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:02:18.041888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed11bf-7d32-43d2-a2c6-744c84649b1d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:02:18.052607 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed11bf-7d32-43d2-a2c6-744c84649b1d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:02:18.104671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:02:18.111556 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732baea-a081-4597-9b30-71820a21f4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:02:18.112166 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732baea-a081-4597-9b30-71820a21f4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:02:18.112458 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732baea-a081-4597-9b30-71820a21f4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:02:18.112851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732baea-a081-4597-9b30-71820a21f4f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:02:18.113428 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2732baea-a081-4597-9b30-71820a21f4f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:02:18.114486 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-2732baea-a081-4597-9b30-71820a21f4f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 12:02:18.114486 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Sep 14 12:02:18.114486 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: Sep 14 12:02:18.115721 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2732baea-a081-4597-9b30-71820a21f4f4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:02:18.195828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 751/1498] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:02:18 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:02:18.199971 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d304f50-4c5b-4e81-bc3c-5d77b3e8eb0a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:02:18.210806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d304f50-4c5b-4e81-bc3c-5d77b3e8eb0a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:02:18.215979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971683c-f8ef-44f7-8a13-a3163597690d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:02:18.216638 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971683c-f8ef-44f7-8a13-a3163597690d admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:02:18.216896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971683c-f8ef-44f7-8a13-a3163597690d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:02:18.217170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971683c-f8ef-44f7-8a13-a3163597690d admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:02:18.217812 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971683c-f8ef-44f7-8a13-a3163597690d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:02:18.218848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-e971683c-f8ef-44f7-8a13-a3163597690d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 14 12:02:18.218848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Sep 14 12:02:18.218848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: Sep 14 12:02:18.219728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971683c-f8ef-44f7-8a13-a3163597690d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:02:18.232431 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 748/1499] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:02:18.236598 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732baea-a081-4597-9b30-71820a21f4f4 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:02:18.246699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732baea-a081-4597-9b30-71820a21f4f4 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:02:18.251882 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44939f9d-203e-469c-947c-fae346143684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:02:18.252487 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44939f9d-203e-469c-947c-fae346143684 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:02:18.252771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44939f9d-203e-469c-947c-fae346143684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:02:18.253059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44939f9d-203e-469c-947c-fae346143684 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:02:18.253728 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44939f9d-203e-469c-947c-fae346143684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:02:18.254762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-44939f9d-203e-469c-947c-fae346143684 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 14 12:02:18.254762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Sep 14 12:02:18.254762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: Sep 14 12:02:18.255642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44939f9d-203e-469c-947c-fae346143684 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:02:18.269430 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 752/1500] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:02:18.273482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971683c-f8ef-44f7-8a13-a3163597690d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:02:18.283570 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971683c-f8ef-44f7-8a13-a3163597690d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:02:18.288368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb09173-6b56-4f2b-a265-add6188c04d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:02:18.289020 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb09173-6b56-4f2b-a265-add6188c04d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:02:18.289301 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb09173-6b56-4f2b-a265-add6188c04d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:02:18.289576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb09173-6b56-4f2b-a265-add6188c04d7 admin admin] PATH_INFO: `/v3/projects/fb95bb689b074454aee9c0220c42dcfc/users/f700db07422d41d38616dee6f59beebe/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:02:18.304178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb09173-6b56-4f2b-a265-add6188c04d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb95bb689b074454aee9c0220c42dcfc, user_id=f700db07422d41d38616dee6f59beebe, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:02:18.305205 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-4fb09173-6b56-4f2b-a265-add6188c04d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 12:02:18.305205 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Sep 14 12:02:18.305205 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: Sep 14 12:02:18.307157 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb09173-6b56-4f2b-a265-add6188c04d7 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:02:18.390405 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 749/1501] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:02:18 2022] PUT /identity/v3/projects/fb95bb689b074454aee9c0220c42dcfc/users/f700db07422d41d38616dee6f59beebe/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:02:18.402539 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240b73e-c5d1-4932-805b-a5ed38456fd1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:02:18.404961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240b73e-c5d1-4932-805b-a5ed38456fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:02:18.404961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240b73e-c5d1-4932-805b-a5ed38456fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:02:18.431578 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9240b73e-c5d1-4932-805b-a5ed38456fd1 None None] MFA Rules not processed for user `f700db07422d41d38616dee6f59beebe`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:02:18.433595 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9240b73e-c5d1-4932-805b-a5ed38456fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:02:18.498954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 753/1502] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:02:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:02:18.512108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb3705-3d01-4dcc-bcac-88ca4bc083b5 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:02:18.512436 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb3705-3d01-4dcc-bcac-88ca4bc083b5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:02:18.512732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb3705-3d01-4dcc-bcac-88ca4bc083b5 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:02:18.514278 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 750/1503] 10.222.0.62 () {54 vars in 964 bytes} [Wed Sep 14 12:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:02:18.519464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26834876-7fb8-4c2f-9c14-481cedc763e4 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:02:18.519756 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26834876-7fb8-4c2f-9c14-481cedc763e4 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:02:18.520036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26834876-7fb8-4c2f-9c14-481cedc763e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:02:18.582238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-26834876-7fb8-4c2f-9c14-481cedc763e4 None None] MFA Rules not processed for user `d27c7ff726e443e89da94f282b5cddab`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:02:18.583477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-26834876-7fb8-4c2f-9c14-481cedc763e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:02:18.660769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 754/1504] 10.222.0.62 () {58 vars in 1061 bytes} [Wed Sep 14 12:02:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:02:18.671039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb3705-3d01-4dcc-bcac-88ca4bc083b5 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:02:18.682183 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb3705-3d01-4dcc-bcac-88ca4bc083b5 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:02:18.686479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ae268-bf61-450b-838d-817ca3a4edb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:02:18.687079 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ae268-bf61-450b-838d-817ca3a4edb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:02:18.687361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ae268-bf61-450b-838d-817ca3a4edb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:02:18.687633 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ae268-bf61-450b-838d-817ca3a4edb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:02:18.696028 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ae268-bf61-450b-838d-817ca3a4edb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:02:18.697457 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ae268-bf61-450b-838d-817ca3a4edb0 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:02:18.706474 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 751/1505] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:02:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:02:18.756880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025cb3a-01b4-4446-b869-ea4eb52ee251 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:02:18.757176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025cb3a-01b4-4446-b869-ea4eb52ee251 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:02:18.757466 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025cb3a-01b4-4446-b869-ea4eb52ee251 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:02:18.758864 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 755/1506] 10.222.0.62 () {52 vars in 830 bytes} [Wed Sep 14 12:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:02:18.762504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ae268-bf61-450b-838d-817ca3a4edb0 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:02:18.826700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ae268-bf61-450b-838d-817ca3a4edb0 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:02:18.833391 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37a575-53e1-427c-8d61-97fea8f19d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:02:18.834082 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37a575-53e1-427c-8d61-97fea8f19d4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:02:18.834365 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37a575-53e1-427c-8d61-97fea8f19d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:02:18.834641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37a575-53e1-427c-8d61-97fea8f19d4b admin admin] PATH_INFO: `/v3/users/f700db07422d41d38616dee6f59beebe` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:02:18.835838 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37a575-53e1-427c-8d61-97fea8f19d4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f700db07422d41d38616dee6f59beebe)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:02:18.836874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-de37a575-53e1-427c-8d61-97fea8f19d4b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 12:02:18.836874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Sep 14 12:02:18.836874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: Sep 14 12:02:18.838120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37a575-53e1-427c-8d61-97fea8f19d4b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:02:19.065109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-de37a575-53e1-427c-8d61-97fea8f19d4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f700db07422d41d38616dee6f59beebe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:02:19.089778 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-de37a575-53e1-427c-8d61-97fea8f19d4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f700db07422d41d38616dee6f59beebe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:02:19.106859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-de37a575-53e1-427c-8d61-97fea8f19d4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f700db07422d41d38616dee6f59beebe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:02:19.111542 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:02:19.206374 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-de37a575-53e1-427c-8d61-97fea8f19d4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f700db07422d41d38616dee6f59beebe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:02:19.207012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-de37a575-53e1-427c-8d61-97fea8f19d4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f700db07422d41d38616dee6f59beebe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:02:19.213490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 752/1507] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:02:18 2022] DELETE /identity/v3/users/f700db07422d41d38616dee6f59beebe => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:02:19.218762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2167ca-1c62-46c5-a81f-d9783a5b9389 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:02:19.220437 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2167ca-1c62-46c5-a81f-d9783a5b9389 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:02:19.220437 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2167ca-1c62-46c5-a81f-d9783a5b9389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:02:19.238675 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4b2167ca-1c62-46c5-a81f-d9783a5b9389 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:02:19.239886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4b2167ca-1c62-46c5-a81f-d9783a5b9389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:02:19.249327 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:02:19.300032 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 756/1508] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:02:19.309713 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37a575-53e1-427c-8d61-97fea8f19d4b admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:02:19.311181 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-de37a575-53e1-427c-8d61-97fea8f19d4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:02:19.387127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37a575-53e1-427c-8d61-97fea8f19d4b admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:02:19.391405 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dac1ef-0d46-436a-be06-9129a3776c24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:02:19.392011 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dac1ef-0d46-436a-be06-9129a3776c24 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:02:19.392292 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dac1ef-0d46-436a-be06-9129a3776c24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:02:19.392566 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dac1ef-0d46-436a-be06-9129a3776c24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:02:19.401382 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dac1ef-0d46-436a-be06-9129a3776c24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:02:19.402821 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dac1ef-0d46-436a-be06-9129a3776c24 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:02:19.412853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 753/1509] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:02:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:02:19.871643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2167ca-1c62-46c5-a81f-d9783a5b9389 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:02:19.873597 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4b2167ca-1c62-46c5-a81f-d9783a5b9389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:02:19.885778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2167ca-1c62-46c5-a81f-d9783a5b9389 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:02:19.891135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a946e-1be6-4458-b3f3-06688c84d0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:02:19.891791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a946e-1be6-4458-b3f3-06688c84d0c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:02:19.892077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a946e-1be6-4458-b3f3-06688c84d0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:02:19.892356 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a946e-1be6-4458-b3f3-06688c84d0c4 admin admin] PATH_INFO: `/v3/projects/fb95bb689b074454aee9c0220c42dcfc` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:02:19.893517 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a946e-1be6-4458-b3f3-06688c84d0c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb95bb689b074454aee9c0220c42dcfc)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:02:19.894695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-cf3a946e-1be6-4458-b3f3-06688c84d0c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 12:02:19.894695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Sep 14 12:02:19.894695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: Sep 14 12:02:19.896646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a946e-1be6-4458-b3f3-06688c84d0c4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:02:20.047922 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-cf3a946e-1be6-4458-b3f3-06688c84d0c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb95bb689b074454aee9c0220c42dcfc', 'request_id': 'req-cf3a946e-1be6-4458-b3f3-06688c84d0c4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:02:20.137839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-cf3a946e-1be6-4458-b3f3-06688c84d0c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb95bb689b074454aee9c0220c42dcfc', 'request_id': 'req-cf3a946e-1be6-4458-b3f3-06688c84d0c4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:02:20.144959 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-cf3a946e-1be6-4458-b3f3-06688c84d0c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb95bb689b074454aee9c0220c42dcfc', 'request_id': 'req-cf3a946e-1be6-4458-b3f3-06688c84d0c4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:02:20.164076 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-cf3a946e-1be6-4458-b3f3-06688c84d0c4 admin admin] The token cache is being invalidate because project fb95bb689b074454aee9c0220c42dcfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:02:20.164370 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-cf3a946e-1be6-4458-b3f3-06688c84d0c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:02:20.165072 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-cf3a946e-1be6-4458-b3f3-06688c84d0c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:02:20.166725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 757/1510] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:02:19 2022] DELETE /identity/v3/projects/fb95bb689b074454aee9c0220c42dcfc => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:02:29.408733 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:02:30.161004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:03:52.771990 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053134dd-420d-4df6-94b3-ca08ae84ada7 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:03:52.772628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053134dd-420d-4df6-94b3-ca08ae84ada7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:03:52.772628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053134dd-420d-4df6-94b3-ca08ae84ada7 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:03:52.773905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 12:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:03:52.779196 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e19bc-a1b7-4fc8-8546-abc1d4a34148 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:03:52.779623 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e19bc-a1b7-4fc8-8546-abc1d4a34148 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:03:52.779769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e19bc-a1b7-4fc8-8546-abc1d4a34148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:03:52.801709 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b10e19bc-a1b7-4fc8-8546-abc1d4a34148 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:03:52.803570 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b10e19bc-a1b7-4fc8-8546-abc1d4a34148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:03:52.984827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 12:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:03:53.138071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053134dd-420d-4df6-94b3-ca08ae84ada7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:03:53.139870 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-053134dd-420d-4df6-94b3-ca08ae84ada7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:03:53.241751 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053134dd-420d-4df6-94b3-ca08ae84ada7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:03:53.247268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa1e79-8b07-4ded-8d80-ef9ed1c41c8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:03:53.247871 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa1e79-8b07-4ded-8d80-ef9ed1c41c8d service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:03:53.248153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa1e79-8b07-4ded-8d80-ef9ed1c41c8d service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:03:53.248427 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa1e79-8b07-4ded-8d80-ef9ed1c41c8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:03:53.256656 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa1e79-8b07-4ded-8d80-ef9ed1c41c8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:03:53.258266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa1e79-8b07-4ded-8d80-ef9ed1c41c8d service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:03:53.267393 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 755/1513] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:03:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:03:54.359509 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2cb70-381c-4a3b-a77a-bfb3f327f568 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:03:54.360087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2cb70-381c-4a3b-a77a-bfb3f327f568 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:03:54.360087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2cb70-381c-4a3b-a77a-bfb3f327f568 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:03:54.361446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 12:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:03:54.367860 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9a64f-a099-4c6d-b3e6-c6b18279760e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:03:54.368285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9a64f-a099-4c6d-b3e6-c6b18279760e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:03:54.368430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9a64f-a099-4c6d-b3e6-c6b18279760e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:03:54.390110 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-01d9a64f-a099-4c6d-b3e6-c6b18279760e None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:03:54.391326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-01d9a64f-a099-4c6d-b3e6-c6b18279760e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:03:54.399694 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 12:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:03:54.550829 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2cb70-381c-4a3b-a77a-bfb3f327f568 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:03:54.565248 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2cb70-381c-4a3b-a77a-bfb3f327f568 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:03:54.571330 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a3328-37e2-4a7a-afd2-04d0e0fe3f4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:03:54.571931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a3328-37e2-4a7a-afd2-04d0e0fe3f4a service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:03:54.572211 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a3328-37e2-4a7a-afd2-04d0e0fe3f4a service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:03:54.572485 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a3328-37e2-4a7a-afd2-04d0e0fe3f4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:03:54.580854 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a3328-37e2-4a7a-afd2-04d0e0fe3f4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:03:54.582476 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a3328-37e2-4a7a-afd2-04d0e0fe3f4a service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:03:54.592425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 760/1516] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:03:55.501724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2a927-c2a9-4d70-8624-8edf9bdc9f13 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:03:55.502373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2a927-c2a9-4d70-8624-8edf9bdc9f13 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:03:55.502373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2a927-c2a9-4d70-8624-8edf9bdc9f13 None None] PATH_INFO: `/v3` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:03:55.503568 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 757/1517] 10.222.0.81 () {54 vars in 925 bytes} [Wed Sep 14 12:03:55 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 12:03:55.510983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be40e3-3a21-4e94-8116-453300d21c68 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:03:55.511433 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be40e3-3a21-4e94-8116-453300d21c68 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:03:55.511531 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be40e3-3a21-4e94-8116-453300d21c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:03:55.563874 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a5be40e3-3a21-4e94-8116-453300d21c68 None None] MFA Rules not processed for user `0ca10dd7ed714edb8b87017deecc41b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:03:55.565135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a5be40e3-3a21-4e94-8116-453300d21c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:03:55.658036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 761/1518] 10.222.0.81 () {58 vars in 1015 bytes} [Wed Sep 14 12:03:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:04:04.382563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=49 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:04:05.653894 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=77 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:04:05.789728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c45236-a391-4650-8014-ba31537a38e2 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:05.790309 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c45236-a391-4650-8014-ba31537a38e2 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:05.790394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c45236-a391-4650-8014-ba31537a38e2 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:05.791717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 758/1519] 10.222.0.62 () {54 vars in 961 bytes} [Wed Sep 14 12:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:04:05.797097 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff49f57-2b87-4300-a554-0832374ca6f5 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:05.797397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff49f57-2b87-4300-a554-0832374ca6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:05.797683 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff49f57-2b87-4300-a554-0832374ca6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:05.818274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-aff49f57-2b87-4300-a554-0832374ca6f5 None None] MFA Rules not processed for user `0ca10dd7ed714edb8b87017deecc41b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:04:05.819539 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-aff49f57-2b87-4300-a554-0832374ca6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:04:05.829551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 762/1520] 10.222.0.62 () {58 vars in 1058 bytes} [Wed Sep 14 12:04:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:04:05.836445 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c45236-a391-4650-8014-ba31537a38e2 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:04:05.851717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c45236-a391-4650-8014-ba31537a38e2 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:04:05.857586 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3ef3b-a901-4754-a803-a6a6c8d11bcd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca10dd7ed714edb8b87017deecc41b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:04:05.858294 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3ef3b-a901-4754-a803-a6a6c8d11bcd service placement] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:05.858676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3ef3b-a901-4754-a803-a6a6c8d11bcd service placement] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:05.858953 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3ef3b-a901-4754-a803-a6a6c8d11bcd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:05.868622 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3ef3b-a901-4754-a803-a6a6c8d11bcd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:04:05.870361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3ef3b-a901-4754-a803-a6a6c8d11bcd service placement] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:04:05.887596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 759/1521] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:04:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:04:06.108588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97055c-8a9b-489d-b024-ddb9ba8a0c44 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:06.108876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97055c-8a9b-489d-b024-ddb9ba8a0c44 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:06.109154 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97055c-8a9b-489d-b024-ddb9ba8a0c44 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:06.110682 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 12:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 12:04:06.117915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8fcf62-1ed2-4fae-a94f-b93fd350b718 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:06.118184 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8fcf62-1ed2-4fae-a94f-b93fd350b718 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:06.118460 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8fcf62-1ed2-4fae-a94f-b93fd350b718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:06.138532 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ee8fcf62-1ed2-4fae-a94f-b93fd350b718 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:04:06.139736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ee8fcf62-1ed2-4fae-a94f-b93fd350b718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:04:06.148225 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 12:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:04:06.302002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97055c-8a9b-489d-b024-ddb9ba8a0c44 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:04:06.317536 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97055c-8a9b-489d-b024-ddb9ba8a0c44 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:04:06.324186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cbad62-e098-444e-b83d-56760da940a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:04:06.324806 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbad62-e098-444e-b83d-56760da940a0 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:06.325100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbad62-e098-444e-b83d-56760da940a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:06.325405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbad62-e098-444e-b83d-56760da940a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:06.337373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cbad62-e098-444e-b83d-56760da940a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:04:06.340556 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cbad62-e098-444e-b83d-56760da940a0 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:04:06.349745 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 764/1524] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:04:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:04:07.458252 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0b547-f568-425c-a062-cc03c630e4da None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:07.458805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0b547-f568-425c-a062-cc03c630e4da None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:07.458805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0b547-f568-425c-a062-cc03c630e4da None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:07.460217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 12:04:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:04:07.466572 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814e790-c062-4967-bbce-66ba60d9fd6b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:07.467031 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814e790-c062-4967-bbce-66ba60d9fd6b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:07.467140 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814e790-c062-4967-bbce-66ba60d9fd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:07.487070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-5814e790-c062-4967-bbce-66ba60d9fd6b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:04:07.488278 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5814e790-c062-4967-bbce-66ba60d9fd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:04:07.496001 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 12:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:04:07.649481 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0b547-f568-425c-a062-cc03c630e4da None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:04:07.664888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0b547-f568-425c-a062-cc03c630e4da None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:04:07.670356 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a237d-123b-483b-8791-f2b5a77e357c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:04:07.670961 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a237d-123b-483b-8791-f2b5a77e357c service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:07.671245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a237d-123b-483b-8791-f2b5a77e357c service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:07.671522 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a237d-123b-483b-8791-f2b5a77e357c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:07.680051 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a237d-123b-483b-8791-f2b5a77e357c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:04:07.681623 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a237d-123b-483b-8791-f2b5a77e357c service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:04:07.691720 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 762/1527] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:04:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:04:11.915263 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20652f-8113-43b5-a226-9da94cfd6acc None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:11.915812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20652f-8113-43b5-a226-9da94cfd6acc None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:11.916210 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20652f-8113-43b5-a226-9da94cfd6acc None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:11.918055 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 12:04:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:04:11.924080 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8ec34-a3fc-4f4b-a6f6-2fd5d69cdbc9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:11.924592 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8ec34-a3fc-4f4b-a6f6-2fd5d69cdbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:11.924707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8ec34-a3fc-4f4b-a6f6-2fd5d69cdbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:11.946310 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-89b8ec34-a3fc-4f4b-a6f6-2fd5d69cdbc9 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:04:11.947447 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-89b8ec34-a3fc-4f4b-a6f6-2fd5d69cdbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:04:11.955618 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 12:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:04:12.250479 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20652f-8113-43b5-a226-9da94cfd6acc None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:04:12.252237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2f20652f-8113-43b5-a226-9da94cfd6acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:04:12.312694 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20652f-8113-43b5-a226-9da94cfd6acc None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:04:12.316326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf28cf7-abc2-46c1-abed-3b4fcaa4130b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:04:12.316974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf28cf7-abc2-46c1-abed-3b4fcaa4130b service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:12.317259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf28cf7-abc2-46c1-abed-3b4fcaa4130b service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:12.317535 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf28cf7-abc2-46c1-abed-3b4fcaa4130b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:12.327622 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf28cf7-abc2-46c1-abed-3b4fcaa4130b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:04:12.329075 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf28cf7-abc2-46c1-abed-3b4fcaa4130b service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:04:12.340023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 767/1530] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:04:14.553071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6589f77-c170-4f1f-a6ff-7843a16e31bd None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:14.553655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6589f77-c170-4f1f-a6ff-7843a16e31bd None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:14.553836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6589f77-c170-4f1f-a6ff-7843a16e31bd None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:14.555415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 12:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:04:14.562689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11816dd8-449d-4f29-9018-d93e1df2996a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:14.563238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11816dd8-449d-4f29-9018-d93e1df2996a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:14.563238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11816dd8-449d-4f29-9018-d93e1df2996a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:14.583658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-11816dd8-449d-4f29-9018-d93e1df2996a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:04:14.584893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-11816dd8-449d-4f29-9018-d93e1df2996a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:04:14.592913 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 12:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:04:14.884741 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6589f77-c170-4f1f-a6ff-7843a16e31bd None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:04:14.895997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6589f77-c170-4f1f-a6ff-7843a16e31bd None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:04:14.899907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b92c7c-bf60-4dc4-940b-8d6a0bcac65c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:04:14.900512 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b92c7c-bf60-4dc4-940b-8d6a0bcac65c service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:14.900796 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b92c7c-bf60-4dc4-940b-8d6a0bcac65c service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:14.901073 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b92c7c-bf60-4dc4-940b-8d6a0bcac65c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:14.912489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b92c7c-bf60-4dc4-940b-8d6a0bcac65c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:04:14.913965 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b92c7c-bf60-4dc4-940b-8d6a0bcac65c service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:04:14.923755 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 765/1533] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:04:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:04:18.803542 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11457843-a4bd-452b-81b7-e615912f277d None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:18.804159 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11457843-a4bd-452b-81b7-e615912f277d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:18.804159 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11457843-a4bd-452b-81b7-e615912f277d None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:18.805562 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 12:04:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:04:18.811114 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc82dd8-8fb4-4596-82b3-59363ea9fb91 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:18.811745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc82dd8-8fb4-4596-82b3-59363ea9fb91 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:18.811745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc82dd8-8fb4-4596-82b3-59363ea9fb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:18.836200 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1cc82dd8-8fb4-4596-82b3-59363ea9fb91 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:04:18.837419 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1cc82dd8-8fb4-4596-82b3-59363ea9fb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:04:18.845846 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 12:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:04:19.151780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11457843-a4bd-452b-81b7-e615912f277d None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:04:19.153788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-11457843-a4bd-452b-81b7-e615912f277d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:04:19.214704 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11457843-a4bd-452b-81b7-e615912f277d None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:04:19.218333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a3ef3-3cb4-4608-a8cf-cd57b8443a6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:04:19.218954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a3ef3-3cb4-4608-a8cf-cd57b8443a6e service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:19.219241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a3ef3-3cb4-4608-a8cf-cd57b8443a6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:19.219518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a3ef3-3cb4-4608-a8cf-cd57b8443a6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:19.229462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a3ef3-3cb4-4608-a8cf-cd57b8443a6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:04:19.230919 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a3ef3-3cb4-4608-a8cf-cd57b8443a6e service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:04:19.240851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 770/1536] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:04:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:04:21.415743 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d147b-d56b-4d11-82b2-336cd79582f7 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:21.416663 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d147b-d56b-4d11-82b2-336cd79582f7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:21.416663 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d147b-d56b-4d11-82b2-336cd79582f7 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:21.418280 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 12:04:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:04:21.426044 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11155373-b3b4-43e6-be8b-323bf2b7a3d8 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:21.426559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11155373-b3b4-43e6-be8b-323bf2b7a3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:21.426832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11155373-b3b4-43e6-be8b-323bf2b7a3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:21.450697 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-11155373-b3b4-43e6-be8b-323bf2b7a3d8 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:04:21.451905 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-11155373-b3b4-43e6-be8b-323bf2b7a3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:04:21.461062 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 12:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:04:21.765190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d147b-d56b-4d11-82b2-336cd79582f7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:04:21.768779 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-491d147b-d56b-4d11-82b2-336cd79582f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:04:21.781730 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d147b-d56b-4d11-82b2-336cd79582f7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:04:21.788507 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839de19-b23d-4c89-8d9d-3343a636d237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:04:21.789066 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839de19-b23d-4c89-8d9d-3343a636d237 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:04:21.789350 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839de19-b23d-4c89-8d9d-3343a636d237 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:04:21.789776 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839de19-b23d-4c89-8d9d-3343a636d237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:04:21.799584 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a839de19-b23d-4c89-8d9d-3343a636d237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:04:21.803445 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a839de19-b23d-4c89-8d9d-3343a636d237 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:04:21.810797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 768/1539] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:04:31.445628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=70 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:04:31.806486 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=33 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:49.210524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc30c43-e7fb-432e-9935-c1a5aae6b85c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:49.211217 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc30c43-e7fb-432e-9935-c1a5aae6b85c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:49.211521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc30c43-e7fb-432e-9935-c1a5aae6b85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:49.251440 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8dc30c43-e7fb-432e-9935-c1a5aae6b85c None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:49.252836 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8dc30c43-e7fb-432e-9935-c1a5aae6b85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:49.269416 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 772/1540] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:49.279267 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42292740-2855-4ffc-87e8-96c1ba21c823 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:49.280393 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42292740-2855-4ffc-87e8-96c1ba21c823 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:49.280881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42292740-2855-4ffc-87e8-96c1ba21c823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:49.312880 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-42292740-2855-4ffc-87e8-96c1ba21c823 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:49.314509 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-42292740-2855-4ffc-87e8-96c1ba21c823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:49.335466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 769/1541] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:49.344218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc30c43-e7fb-432e-9935-c1a5aae6b85c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:49.361704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dd3d1-0039-464a-85f8-31179e3e0c02 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:49.362023 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dd3d1-0039-464a-85f8-31179e3e0c02 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:49.362364 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dd3d1-0039-464a-85f8-31179e3e0c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:49.373158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc30c43-e7fb-432e-9935-c1a5aae6b85c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:49.384880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738649b9-78c2-4893-81ec-5086206daea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:49.385529 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738649b9-78c2-4893-81ec-5086206daea8 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:49.385817 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738649b9-78c2-4893-81ec-5086206daea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:49.386123 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738649b9-78c2-4893-81ec-5086206daea8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:49.386806 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738649b9-78c2-4893-81ec-5086206daea8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:05:49.387118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738649b9-78c2-4893-81ec-5086206daea8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:49.388774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738649b9-78c2-4893-81ec-5086206daea8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:49.418873 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-037dd3d1-0039-464a-85f8-31179e3e0c02 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:49.420637 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-037dd3d1-0039-464a-85f8-31179e3e0c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:49.438771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 773/1542] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:05:49.444065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 770/1543] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:05:49.454612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240a472-4126-4b3e-bce0-2f16f5e0d7de None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:49.454847 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738649b9-78c2-4893-81ec-5086206daea8 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:49.454950 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240a472-4126-4b3e-bce0-2f16f5e0d7de None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:49.455184 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240a472-4126-4b3e-bce0-2f16f5e0d7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:49.485097 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738649b9-78c2-4893-81ec-5086206daea8 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:49.501361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4240a472-4126-4b3e-bce0-2f16f5e0d7de None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:49.503132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4240a472-4126-4b3e-bce0-2f16f5e0d7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:49.508879 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd676a-8a15-48bb-937e-1bacedb2558b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:49.509826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd676a-8a15-48bb-937e-1bacedb2558b admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:49.510308 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd676a-8a15-48bb-937e-1bacedb2558b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:49.510852 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd676a-8a15-48bb-937e-1bacedb2558b admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:49.511697 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd676a-8a15-48bb-937e-1bacedb2558b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:49.512913 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-95fd676a-8a15-48bb-937e-1bacedb2558b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 12:05:49.512913 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Sep 14 12:05:49.512913 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: Sep 14 12:05:49.514646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd676a-8a15-48bb-937e-1bacedb2558b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:49.525918 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 771/1544] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:49.527811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47423726-e431-44d8-80a7-972090eb69dd None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:49.528326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47423726-e431-44d8-80a7-972090eb69dd None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:49.528848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47423726-e431-44d8-80a7-972090eb69dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:49.570930 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-47423726-e431-44d8-80a7-972090eb69dd None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:49.572792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-47423726-e431-44d8-80a7-972090eb69dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:49.588077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 772/1545] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:49.590020 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367e156-56c5-49a2-b89e-ea347ffb0cd0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:49.594681 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367e156-56c5-49a2-b89e-ea347ffb0cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:49.594681 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367e156-56c5-49a2-b89e-ea347ffb0cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:49.620331 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1367e156-56c5-49a2-b89e-ea347ffb0cd0 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:49.621893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1367e156-56c5-49a2-b89e-ea347ffb0cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:49.636069 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 773/1546] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:49.637417 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367e156-56c5-49a2-b89e-ea347ffb0cd0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:49.649536 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367e156-56c5-49a2-b89e-ea347ffb0cd0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:49.655076 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c9f6d-5afb-4af4-9738-c4dd4d730a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:49.655700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c9f6d-5afb-4af4-9738-c4dd4d730a54 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:49.655985 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c9f6d-5afb-4af4-9738-c4dd4d730a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:49.656260 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c9f6d-5afb-4af4-9738-c4dd4d730a54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:49.656923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c9f6d-5afb-4af4-9738-c4dd4d730a54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:05:49.657295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c9f6d-5afb-4af4-9738-c4dd4d730a54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:49.658336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-303c9f6d-5afb-4af4-9738-c4dd4d730a54 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 14 12:05:49.658336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Sep 14 12:05:49.658336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: Sep 14 12:05:49.659233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c9f6d-5afb-4af4-9738-c4dd4d730a54 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:49.672765 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 774/1547] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:05:49.674242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdc086-2c39-4806-a8cf-45fb8442ee2f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:49.674523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdc086-2c39-4806-a8cf-45fb8442ee2f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:49.674801 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdc086-2c39-4806-a8cf-45fb8442ee2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:49.701483 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 774/1548] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:05:49 2022] POST /identity/v3/projects => generated 389 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:49.701483 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff63fb5-0df6-4540-9da5-b1c765b3c6ca None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:49.701483 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff63fb5-0df6-4540-9da5-b1c765b3c6ca None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:49.701483 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff63fb5-0df6-4540-9da5-b1c765b3c6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:49.708449 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-fcbdc086-2c39-4806-a8cf-45fb8442ee2f None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:49.709747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fcbdc086-2c39-4806-a8cf-45fb8442ee2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:49.741770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-eff63fb5-0df6-4540-9da5-b1c765b3c6ca None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:49.743106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-eff63fb5-0df6-4540-9da5-b1c765b3c6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:49.780529 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 775/1549] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:49.783177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f8e8c-5e75-4a2c-bb5c-2e660f66cd3f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:49.783479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f8e8c-5e75-4a2c-bb5c-2e660f66cd3f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:49.783828 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f8e8c-5e75-4a2c-bb5c-2e660f66cd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:49.804343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d87f8e8c-5e75-4a2c-bb5c-2e660f66cd3f None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:49.805570 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d87f8e8c-5e75-4a2c-bb5c-2e660f66cd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:49.813759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 776/1550] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:49.815346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6df86d-7e17-4700-9c08-a1c3ecee0be2 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:49.815647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6df86d-7e17-4700-9c08-a1c3ecee0be2 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:49.816489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6df86d-7e17-4700-9c08-a1c3ecee0be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:49.836039 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 775/1551] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:49.837326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e91a17-a6b0-4443-873c-846b4dad98b8 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:49.837617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e91a17-a6b0-4443-873c-846b4dad98b8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:49.837928 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e91a17-a6b0-4443-873c-846b4dad98b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:49.844525 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ef6df86d-7e17-4700-9c08-a1c3ecee0be2 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:49.845741 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ef6df86d-7e17-4700-9c08-a1c3ecee0be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:49.854940 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 777/1552] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:49.855726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6df86d-7e17-4700-9c08-a1c3ecee0be2 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:49.858394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d3e91a17-a6b0-4443-873c-846b4dad98b8 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:49.859600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d3e91a17-a6b0-4443-873c-846b4dad98b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:49.868420 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 776/1553] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:49.869278 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e91a17-a6b0-4443-873c-846b4dad98b8 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:49.871905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6df86d-7e17-4700-9c08-a1c3ecee0be2 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:49.876866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301494a9-bcb7-4a58-8ff3-14a2935ad3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:49.877558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301494a9-bcb7-4a58-8ff3-14a2935ad3db admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:49.877993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301494a9-bcb7-4a58-8ff3-14a2935ad3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:49.878175 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301494a9-bcb7-4a58-8ff3-14a2935ad3db admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:49.879465 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301494a9-bcb7-4a58-8ff3-14a2935ad3db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:49.881237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e91a17-a6b0-4443-873c-846b4dad98b8 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:49.881363 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301494a9-bcb7-4a58-8ff3-14a2935ad3db admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:49.885914 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ac1ee-52f6-46a8-8b27-0b579997bca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:49.886582 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ac1ee-52f6-46a8-8b27-0b579997bca1 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:49.886865 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ac1ee-52f6-46a8-8b27-0b579997bca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:49.887142 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ac1ee-52f6-46a8-8b27-0b579997bca1 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:49.887784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ac1ee-52f6-46a8-8b27-0b579997bca1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:49.889723 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ac1ee-52f6-46a8-8b27-0b579997bca1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:49.997297 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 777/1554] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:05:49 2022] POST /identity/v3/users => generated 295 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:49.998485 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c187b-8671-4e06-8bdc-3b4da9d8acb9 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:49.998776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c187b-8671-4e06-8bdc-3b4da9d8acb9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:49.999050 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c187b-8671-4e06-8bdc-3b4da9d8acb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.011785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 778/1555] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:05:49 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:50.013612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2664de5-95dd-49d5-811d-08d8d21efe2f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.014085 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2664de5-95dd-49d5-811d-08d8d21efe2f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.014509 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2664de5-95dd-49d5-811d-08d8d21efe2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.023009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-859c187b-8671-4e06-8bdc-3b4da9d8acb9 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:50.024244 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-859c187b-8671-4e06-8bdc-3b4da9d8acb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:50.040707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e2664de5-95dd-49d5-811d-08d8d21efe2f None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:50.041941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e2664de5-95dd-49d5-811d-08d8d21efe2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:50.099835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 778/1556] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:50.101011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c187b-8671-4e06-8bdc-3b4da9d8acb9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:50.109488 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 779/1557] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:50.111098 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394262d-8b48-48cd-a7fb-bff750bf8b30 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.111390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394262d-8b48-48cd-a7fb-bff750bf8b30 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.111666 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394262d-8b48-48cd-a7fb-bff750bf8b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.113144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c187b-8671-4e06-8bdc-3b4da9d8acb9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:50.121506 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c71e7-9d32-4d0c-82ed-4ff166a498d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:50.125564 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c71e7-9d32-4d0c-82ed-4ff166a498d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.125910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c71e7-9d32-4d0c-82ed-4ff166a498d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.126194 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c71e7-9d32-4d0c-82ed-4ff166a498d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.126993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c71e7-9d32-4d0c-82ed-4ff166a498d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:05:50.127312 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c71e7-9d32-4d0c-82ed-4ff166a498d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:50.129918 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c71e7-9d32-4d0c-82ed-4ff166a498d7 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:50.134099 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e394262d-8b48-48cd-a7fb-bff750bf8b30 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:50.135385 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e394262d-8b48-48cd-a7fb-bff750bf8b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:50.142261 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 779/1558] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:05:50.143197 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c71e7-9d32-4d0c-82ed-4ff166a498d7 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:50.143571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 780/1559] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:50.145352 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce6b91-e655-47df-ae38-093c5c30e958 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.145627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce6b91-e655-47df-ae38-093c5c30e958 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.145941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce6b91-e655-47df-ae38-093c5c30e958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.162159 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c71e7-9d32-4d0c-82ed-4ff166a498d7 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:50.165295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e2ce6b91-e655-47df-ae38-093c5c30e958 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:50.166511 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e2ce6b91-e655-47df-ae38-093c5c30e958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:50.167449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c77dd6-26d2-4e2c-aa68-e3cb78a6c858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:50.168085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c77dd6-26d2-4e2c-aa68-e3cb78a6c858 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.168370 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c77dd6-26d2-4e2c-aa68-e3cb78a6c858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.168655 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c77dd6-26d2-4e2c-aa68-e3cb78a6c858 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.169297 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c77dd6-26d2-4e2c-aa68-e3cb78a6c858 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:05:50.169605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c77dd6-26d2-4e2c-aa68-e3cb78a6c858 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:50.171394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c77dd6-26d2-4e2c-aa68-e3cb78a6c858 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:50.174869 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 781/1560] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:50.175705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce6b91-e655-47df-ae38-093c5c30e958 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:50.185200 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 780/1561] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:05:50.185696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c77dd6-26d2-4e2c-aa68-e3cb78a6c858 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:50.189954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce6b91-e655-47df-ae38-093c5c30e958 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:50.196543 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c77dd6-26d2-4e2c-aa68-e3cb78a6c858 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:50.196817 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb7bdf-1641-4c59-ae94-d0702f48752d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:50.197466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb7bdf-1641-4c59-ae94-d0702f48752d admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.197714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb7bdf-1641-4c59-ae94-d0702f48752d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.198266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb7bdf-1641-4c59-ae94-d0702f48752d admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.198703 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb7bdf-1641-4c59-ae94-d0702f48752d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:50.200314 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb7bdf-1641-4c59-ae94-d0702f48752d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:50.201935 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:50.203949 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501859c-4c9c-42be-a0b4-d1e2a2b5dcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:50.204555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501859c-4c9c-42be-a0b4-d1e2a2b5dcc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.204837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501859c-4c9c-42be-a0b4-d1e2a2b5dcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.205116 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501859c-4c9c-42be-a0b4-d1e2a2b5dcc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.205750 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501859c-4c9c-42be-a0b4-d1e2a2b5dcc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:50.208268 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501859c-4c9c-42be-a0b4-d1e2a2b5dcc8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:50.217086 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 782/1562] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:50.217086 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b769a5-bfc9-4253-8035-b8d8a1446fc9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.217086 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b769a5-bfc9-4253-8035-b8d8a1446fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.217086 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b769a5-bfc9-4253-8035-b8d8a1446fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.217726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:50.237475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-62b769a5-bfc9-4253-8035-b8d8a1446fc9 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:50.245608 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-62b769a5-bfc9-4253-8035-b8d8a1446fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:50.247109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 783/1563] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:50.247945 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b769a5-bfc9-4253-8035-b8d8a1446fc9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:50.257994 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b769a5-bfc9-4253-8035-b8d8a1446fc9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:50.262507 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42cb87-b014-4b2e-8f20-ee512a2778be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:50.263083 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42cb87-b014-4b2e-8f20-ee512a2778be admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.263362 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42cb87-b014-4b2e-8f20-ee512a2778be admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.263742 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42cb87-b014-4b2e-8f20-ee512a2778be admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.264373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42cb87-b014-4b2e-8f20-ee512a2778be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:05:50.264684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42cb87-b014-4b2e-8f20-ee512a2778be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:50.266288 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42cb87-b014-4b2e-8f20-ee512a2778be admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:50.278414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 784/1564] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:05:50.279155 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42cb87-b014-4b2e-8f20-ee512a2778be admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:50.289052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42cb87-b014-4b2e-8f20-ee512a2778be admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:50.293994 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7d4eb-09ff-44bf-beb6-60a49ef18a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:50.294610 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7d4eb-09ff-44bf-beb6-60a49ef18a4c admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.294894 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7d4eb-09ff-44bf-beb6-60a49ef18a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.295169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7d4eb-09ff-44bf-beb6-60a49ef18a4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.295831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7d4eb-09ff-44bf-beb6-60a49ef18a4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:05:50.296136 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7d4eb-09ff-44bf-beb6-60a49ef18a4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:50.297713 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7d4eb-09ff-44bf-beb6-60a49ef18a4c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:50.311067 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 785/1565] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:05:50.311903 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7d4eb-09ff-44bf-beb6-60a49ef18a4c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:50.322502 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7d4eb-09ff-44bf-beb6-60a49ef18a4c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:50.327610 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890e479-2106-4cad-93cd-af56ab79a1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:50.331742 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890e479-2106-4cad-93cd-af56ab79a1a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.331915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 781/1566] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:05:50 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:50.332330 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890e479-2106-4cad-93cd-af56ab79a1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.332330 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890e479-2106-4cad-93cd-af56ab79a1a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.332330 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890e479-2106-4cad-93cd-af56ab79a1a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:50.332611 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501859c-4c9c-42be-a0b4-d1e2a2b5dcc8 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:50.332715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890e479-2106-4cad-93cd-af56ab79a1a4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:50.344226 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501859c-4c9c-42be-a0b4-d1e2a2b5dcc8 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:50.349625 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a5a91-af43-4d36-bda0-76f381f67fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:50.350265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a5a91-af43-4d36-bda0-76f381f67fff admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.350550 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a5a91-af43-4d36-bda0-76f381f67fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.350823 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a5a91-af43-4d36-bda0-76f381f67fff admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.351475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a5a91-af43-4d36-bda0-76f381f67fff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:05:50.351780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a5a91-af43-4d36-bda0-76f381f67fff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:50.353403 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a5a91-af43-4d36-bda0-76f381f67fff admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:50.365979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 782/1567] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:05:50.366741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a5a91-af43-4d36-bda0-76f381f67fff admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:50.376330 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a5a91-af43-4d36-bda0-76f381f67fff admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:50.381499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafc3c9-9bb8-497e-9769-140653302ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:50.382129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafc3c9-9bb8-497e-9769-140653302ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.382411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafc3c9-9bb8-497e-9769-140653302ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.382692 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafc3c9-9bb8-497e-9769-140653302ee6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.383317 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafc3c9-9bb8-497e-9769-140653302ee6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:50.385195 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafc3c9-9bb8-497e-9769-140653302ee6 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:50.448242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 786/1568] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:05:50 2022] POST /identity/v3/projects => generated 405 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:50.448242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890e479-2106-4cad-93cd-af56ab79a1a4 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:50.507003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 783/1569] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:05:50 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:50.507914 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafc3c9-9bb8-497e-9769-140653302ee6 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:50.508948 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890e479-2106-4cad-93cd-af56ab79a1a4 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:50.566381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f90d03-b808-4f6e-bdfb-ee44e5028e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:50.566995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f90d03-b808-4f6e-bdfb-ee44e5028e6b admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.567279 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f90d03-b808-4f6e-bdfb-ee44e5028e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.567559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f90d03-b808-4f6e-bdfb-ee44e5028e6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.568201 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f90d03-b808-4f6e-bdfb-ee44e5028e6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:50.569809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f90d03-b808-4f6e-bdfb-ee44e5028e6b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:50.571223 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafc3c9-9bb8-497e-9769-140653302ee6 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:50.576093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655726a2-537d-4b19-82b1-f71115069fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:50.576690 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655726a2-537d-4b19-82b1-f71115069fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.577088 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655726a2-537d-4b19-82b1-f71115069fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.577375 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655726a2-537d-4b19-82b1-f71115069fc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.578058 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655726a2-537d-4b19-82b1-f71115069fc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:05:50.578369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655726a2-537d-4b19-82b1-f71115069fc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:50.579981 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655726a2-537d-4b19-82b1-f71115069fc7 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:50.584468 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 787/1570] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:50.585376 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f90d03-b808-4f6e-bdfb-ee44e5028e6b admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:50.593892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 784/1571] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:05:50.594699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655726a2-537d-4b19-82b1-f71115069fc7 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:50.600078 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f90d03-b808-4f6e-bdfb-ee44e5028e6b admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:50.604935 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de110a8d-b6cd-4f85-b985-c1a9368286ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:50.605548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de110a8d-b6cd-4f85-b985-c1a9368286ad admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.605832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de110a8d-b6cd-4f85-b985-c1a9368286ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.605978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655726a2-537d-4b19-82b1-f71115069fc7 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:50.606278 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de110a8d-b6cd-4f85-b985-c1a9368286ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.606945 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de110a8d-b6cd-4f85-b985-c1a9368286ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:50.608892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de110a8d-b6cd-4f85-b985-c1a9368286ad admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:50.610408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ba919-b93e-4522-9d87-3e3e9ce6f22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:50.611001 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ba919-b93e-4522-9d87-3e3e9ce6f22d admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.611289 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ba919-b93e-4522-9d87-3e3e9ce6f22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.611568 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ba919-b93e-4522-9d87-3e3e9ce6f22d admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.612193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ba919-b93e-4522-9d87-3e3e9ce6f22d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:50.614180 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ba919-b93e-4522-9d87-3e3e9ce6f22d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:50.718892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 788/1572] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:05:50 2022] POST /identity/v3/projects => generated 391 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:50.718892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de110a8d-b6cd-4f85-b985-c1a9368286ad admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:50.746119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 785/1573] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:05:50 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:50.747647 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ba919-b93e-4522-9d87-3e3e9ce6f22d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:50.755070 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:50.794305 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de110a8d-b6cd-4f85-b985-c1a9368286ad admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:50.825126 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ba919-b93e-4522-9d87-3e3e9ce6f22d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:50.830391 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153deaf4-9ef5-46ff-bece-3dd06c90b042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:50.831042 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153deaf4-9ef5-46ff-bece-3dd06c90b042 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.832464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153deaf4-9ef5-46ff-bece-3dd06c90b042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.832464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153deaf4-9ef5-46ff-bece-3dd06c90b042 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.833006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153deaf4-9ef5-46ff-bece-3dd06c90b042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:50.835070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153deaf4-9ef5-46ff-bece-3dd06c90b042 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:50.861257 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4d155-de7c-4192-8c12-509e59d014a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:50.861908 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4d155-de7c-4192-8c12-509e59d014a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.862208 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4d155-de7c-4192-8c12-509e59d014a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.862476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4d155-de7c-4192-8c12-509e59d014a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.863140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4d155-de7c-4192-8c12-509e59d014a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:50.864937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4d155-de7c-4192-8c12-509e59d014a1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:50.875469 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 789/1574] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:50.877181 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4d155-de7c-4192-8c12-509e59d014a1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:50.886885 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4d155-de7c-4192-8c12-509e59d014a1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:50.892046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30891933-ee08-47bd-a7be-cdff41d024d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:50.892659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30891933-ee08-47bd-a7be-cdff41d024d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:50.892972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30891933-ee08-47bd-a7be-cdff41d024d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:50.893251 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30891933-ee08-47bd-a7be-cdff41d024d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:50.893996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30891933-ee08-47bd-a7be-cdff41d024d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:50.895527 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-30891933-ee08-47bd-a7be-cdff41d024d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 12:05:50.895527 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Sep 14 12:05:50.895527 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: Sep 14 12:05:50.896773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30891933-ee08-47bd-a7be-cdff41d024d5 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:50.935633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 786/1575] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:05:50 2022] POST /identity/v3/projects => generated 373 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:50.936665 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153deaf4-9ef5-46ff-bece-3dd06c90b042 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:50.950582 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:50.999916 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153deaf4-9ef5-46ff-bece-3dd06c90b042 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.005784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb666c6-52c4-4940-ac2d-320622e5384e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.006467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb666c6-52c4-4940-ac2d-320622e5384e admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.006729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb666c6-52c4-4940-ac2d-320622e5384e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.007010 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb666c6-52c4-4940-ac2d-320622e5384e admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.007681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb666c6-52c4-4940-ac2d-320622e5384e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.009616 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb666c6-52c4-4940-ac2d-320622e5384e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.020343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 790/1576] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:05:50 2022] POST /identity/v3/users => generated 303 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:51.021466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30891933-ee08-47bd-a7be-cdff41d024d5 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.031485 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30891933-ee08-47bd-a7be-cdff41d024d5 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.036127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b5a63-5025-4e7a-b23e-377866dad956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.036780 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b5a63-5025-4e7a-b23e-377866dad956 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.037063 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b5a63-5025-4e7a-b23e-377866dad956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.037345 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b5a63-5025-4e7a-b23e-377866dad956 admin admin] PATH_INFO: `/v3/projects/d410170ed3ed4ce2b64d367cbf77790a/users/28e44067e07d4501a840108f4dcf053c/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.053298 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b5a63-5025-4e7a-b23e-377866dad956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d410170ed3ed4ce2b64d367cbf77790a, user_id=28e44067e07d4501a840108f4dcf053c, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.055928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b5a63-5025-4e7a-b23e-377866dad956 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.123257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 787/1577] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:05:50 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:51.123257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb666c6-52c4-4940-ac2d-320622e5384e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.134762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb666c6-52c4-4940-ac2d-320622e5384e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.141166 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e200b83-0966-4ef1-987e-963a8783e6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.141948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e200b83-0966-4ef1-987e-963a8783e6da admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.142420 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e200b83-0966-4ef1-987e-963a8783e6da admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.142869 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e200b83-0966-4ef1-987e-963a8783e6da admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.143598 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e200b83-0966-4ef1-987e-963a8783e6da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.145527 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e200b83-0966-4ef1-987e-963a8783e6da admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.158796 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 791/1578] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:05:51 2022] PUT /identity/v3/projects/d410170ed3ed4ce2b64d367cbf77790a/users/28e44067e07d4501a840108f4dcf053c/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:05:51.161488 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b5a63-5025-4e7a-b23e-377866dad956 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.229821 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b5a63-5025-4e7a-b23e-377866dad956 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.234772 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c83c2b-e31e-4b3f-9493-08f07197f676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.236011 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c83c2b-e31e-4b3f-9493-08f07197f676 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.236011 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c83c2b-e31e-4b3f-9493-08f07197f676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.236011 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c83c2b-e31e-4b3f-9493-08f07197f676 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.236574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c83c2b-e31e-4b3f-9493-08f07197f676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.239616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c83c2b-e31e-4b3f-9493-08f07197f676 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.245361 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 788/1579] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:05:51 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:51.246888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e200b83-0966-4ef1-987e-963a8783e6da admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.317774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 792/1580] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:05:51 2022] POST /identity/v3/users => generated 296 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:51.318864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c83c2b-e31e-4b3f-9493-08f07197f676 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.319478 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e200b83-0966-4ef1-987e-963a8783e6da admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.325093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6657b5e-6bfd-4751-b284-d681c595c220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.325791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6657b5e-6bfd-4751-b284-d681c595c220 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.326108 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6657b5e-6bfd-4751-b284-d681c595c220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.326467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6657b5e-6bfd-4751-b284-d681c595c220 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.327186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6657b5e-6bfd-4751-b284-d681c595c220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.329344 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6657b5e-6bfd-4751-b284-d681c595c220 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.330167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c83c2b-e31e-4b3f-9493-08f07197f676 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.335315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaaf75-92e9-4d27-9fb8-224f2861fe0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.335923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaaf75-92e9-4d27-9fb8-224f2861fe0d admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.336204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaaf75-92e9-4d27-9fb8-224f2861fe0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.336482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaaf75-92e9-4d27-9fb8-224f2861fe0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.337126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccaaf75-92e9-4d27-9fb8-224f2861fe0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.338786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccaaf75-92e9-4d27-9fb8-224f2861fe0d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.349332 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 793/1581] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:51.350198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaaf75-92e9-4d27-9fb8-224f2861fe0d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.355387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:51.360496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaaf75-92e9-4d27-9fb8-224f2861fe0d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.365745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b6a9b-5441-4b54-9922-5d3c97d1bab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.366583 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b6a9b-5441-4b54-9922-5d3c97d1bab7 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.366766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b6a9b-5441-4b54-9922-5d3c97d1bab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.366944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b6a9b-5441-4b54-9922-5d3c97d1bab7 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.367591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924b6a9b-5441-4b54-9922-5d3c97d1bab7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.369521 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924b6a9b-5441-4b54-9922-5d3c97d1bab7 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.392291 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 789/1582] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:05:51 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:51.393153 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6657b5e-6bfd-4751-b284-d681c595c220 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.403346 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6657b5e-6bfd-4751-b284-d681c595c220 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.408204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961ba63-dd1f-4031-98d4-37810cb10261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.408820 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961ba63-dd1f-4031-98d4-37810cb10261 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.409102 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961ba63-dd1f-4031-98d4-37810cb10261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.409389 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961ba63-dd1f-4031-98d4-37810cb10261 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.410630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961ba63-dd1f-4031-98d4-37810cb10261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.412252 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961ba63-dd1f-4031-98d4-37810cb10261 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.422870 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 790/1583] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:51.423744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961ba63-dd1f-4031-98d4-37810cb10261 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.435975 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961ba63-dd1f-4031-98d4-37810cb10261 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.438116 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 794/1584] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:05:51 2022] POST /identity/v3/users => generated 287 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:51.439389 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f00f42-8851-4da1-9b8f-42d3aec91637 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.439677 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f00f42-8851-4da1-9b8f-42d3aec91637 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.439948 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f00f42-8851-4da1-9b8f-42d3aec91637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.444390 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc243f-d53d-48f9-a725-ce52916c6541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.445013 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc243f-d53d-48f9-a725-ce52916c6541 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.445296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc243f-d53d-48f9-a725-ce52916c6541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.445571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc243f-d53d-48f9-a725-ce52916c6541 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.446254 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc243f-d53d-48f9-a725-ce52916c6541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.447916 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc243f-d53d-48f9-a725-ce52916c6541 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.459763 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 791/1585] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:51.463526 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc243f-d53d-48f9-a725-ce52916c6541 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.464125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-49f00f42-8851-4da1-9b8f-42d3aec91637 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:51.466119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-49f00f42-8851-4da1-9b8f-42d3aec91637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:51.474272 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc243f-d53d-48f9-a725-ce52916c6541 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.475902 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 795/1586] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:51.476742 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f00f42-8851-4da1-9b8f-42d3aec91637 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.481731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72127d82-3deb-4ac8-ac2c-6c6004839038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.482282 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72127d82-3deb-4ac8-ac2c-6c6004839038 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.482589 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72127d82-3deb-4ac8-ac2c-6c6004839038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.482899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72127d82-3deb-4ac8-ac2c-6c6004839038 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.483477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72127d82-3deb-4ac8-ac2c-6c6004839038 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.485594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72127d82-3deb-4ac8-ac2c-6c6004839038 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.488863 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f00f42-8851-4da1-9b8f-42d3aec91637 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.494615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40dd99-9cbd-49c4-8172-f366f6207050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.495178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40dd99-9cbd-49c4-8172-f366f6207050 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.495497 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40dd99-9cbd-49c4-8172-f366f6207050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.495772 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40dd99-9cbd-49c4-8172-f366f6207050 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.496429 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40dd99-9cbd-49c4-8172-f366f6207050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.498095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40dd99-9cbd-49c4-8172-f366f6207050 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.509533 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 796/1587] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:51.510372 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40dd99-9cbd-49c4-8172-f366f6207050 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.521124 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40dd99-9cbd-49c4-8172-f366f6207050 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.529182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342c78c-5c49-4012-a78b-c1ca388e01cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.530114 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342c78c-5c49-4012-a78b-c1ca388e01cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.530585 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342c78c-5c49-4012-a78b-c1ca388e01cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.531053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342c78c-5c49-4012-a78b-c1ca388e01cf admin admin] PATH_INFO: `/v3/projects/e35c2ab9a9644cadadd4da9a544b944b/users/2ba1860ce5504c27816a504776463d9c/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.548160 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342c78c-5c49-4012-a78b-c1ca388e01cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e35c2ab9a9644cadadd4da9a544b944b, user_id=2ba1860ce5504c27816a504776463d9c, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.551516 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342c78c-5c49-4012-a78b-c1ca388e01cf admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.608503 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 792/1588] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:05:51 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:51.612177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72127d82-3deb-4ac8-ac2c-6c6004839038 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.619897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72127d82-3deb-4ac8-ac2c-6c6004839038 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.624806 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff9ac7-843f-4bd0-87d6-5fb905fd997d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.625524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff9ac7-843f-4bd0-87d6-5fb905fd997d admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.625784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff9ac7-843f-4bd0-87d6-5fb905fd997d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.626094 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff9ac7-843f-4bd0-87d6-5fb905fd997d admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.626735 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff9ac7-843f-4bd0-87d6-5fb905fd997d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.628376 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff9ac7-843f-4bd0-87d6-5fb905fd997d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.637052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 797/1589] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:05:51 2022] PUT /identity/v3/projects/e35c2ab9a9644cadadd4da9a544b944b/users/2ba1860ce5504c27816a504776463d9c/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:05:51.643350 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342c78c-5c49-4012-a78b-c1ca388e01cf admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.643556 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 793/1590] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:51.643556 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff9ac7-843f-4bd0-87d6-5fb905fd997d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.655194 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff9ac7-843f-4bd0-87d6-5fb905fd997d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.673146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:51.709994 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342c78c-5c49-4012-a78b-c1ca388e01cf admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.714792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e29924-0fac-4caa-926e-4bfd89947506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.715494 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e29924-0fac-4caa-926e-4bfd89947506 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.715775 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e29924-0fac-4caa-926e-4bfd89947506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.716051 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e29924-0fac-4caa-926e-4bfd89947506 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.716733 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e29924-0fac-4caa-926e-4bfd89947506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.718349 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e29924-0fac-4caa-926e-4bfd89947506 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.722644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc56d2-edcd-434f-ab2f-29223bce99cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.722644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc56d2-edcd-434f-ab2f-29223bce99cc admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.722644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc56d2-edcd-434f-ab2f-29223bce99cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.722644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc56d2-edcd-434f-ab2f-29223bce99cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.723275 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc56d2-edcd-434f-ab2f-29223bce99cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.724830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc56d2-edcd-434f-ab2f-29223bce99cc admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.734770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 798/1591] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:51.736048 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 794/1592] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:51.736607 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e29924-0fac-4caa-926e-4bfd89947506 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.737325 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc56d2-edcd-434f-ab2f-29223bce99cc admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.751349 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e29924-0fac-4caa-926e-4bfd89947506 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.758234 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d0d91-492e-4a56-b30f-cdfcc9f497ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.759124 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d0d91-492e-4a56-b30f-cdfcc9f497ea admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.759413 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d0d91-492e-4a56-b30f-cdfcc9f497ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.759818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d0d91-492e-4a56-b30f-cdfcc9f497ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.760845 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d0d91-492e-4a56-b30f-cdfcc9f497ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.763041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d0d91-492e-4a56-b30f-cdfcc9f497ea admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.778460 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 799/1593] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:51.779673 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d0d91-492e-4a56-b30f-cdfcc9f497ea admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.791221 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d0d91-492e-4a56-b30f-cdfcc9f497ea admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.796229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4c937-fda8-483b-8f76-eef73c5dfa38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.796831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4c937-fda8-483b-8f76-eef73c5dfa38 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.797116 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4c937-fda8-483b-8f76-eef73c5dfa38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.797392 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4c937-fda8-483b-8f76-eef73c5dfa38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.798099 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4c937-fda8-483b-8f76-eef73c5dfa38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.799697 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4c937-fda8-483b-8f76-eef73c5dfa38 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.801277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc56d2-edcd-434f-ab2f-29223bce99cc admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.805289 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678b174-0c88-421d-a0f1-d78777ed63f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.805977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678b174-0c88-421d-a0f1-d78777ed63f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.806266 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678b174-0c88-421d-a0f1-d78777ed63f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.806549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678b174-0c88-421d-a0f1-d78777ed63f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.811508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 800/1594] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:51.812157 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4c937-fda8-483b-8f76-eef73c5dfa38 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.818978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678b174-0c88-421d-a0f1-d78777ed63f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.820491 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678b174-0c88-421d-a0f1-d78777ed63f5 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.821739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4c937-fda8-483b-8f76-eef73c5dfa38 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.827524 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50efca-2e82-4bca-adad-3b07c7df4ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.828196 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50efca-2e82-4bca-adad-3b07c7df4ffa admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.828481 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50efca-2e82-4bca-adad-3b07c7df4ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.828761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50efca-2e82-4bca-adad-3b07c7df4ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.829410 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50efca-2e82-4bca-adad-3b07c7df4ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.831049 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50efca-2e82-4bca-adad-3b07c7df4ffa admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.844871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 795/1595] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:05:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:05:51.845100 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 801/1596] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:51.845237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678b174-0c88-421d-a0f1-d78777ed63f5 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.845329 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c383b0-13c3-4b1d-a576-d547587582a3 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.845329 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c383b0-13c3-4b1d-a576-d547587582a3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.845329 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c383b0-13c3-4b1d-a576-d547587582a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.853716 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678b174-0c88-421d-a0f1-d78777ed63f5 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.860267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f0077-39e5-4d22-a21c-4854745002d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.861087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f0077-39e5-4d22-a21c-4854745002d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.861599 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f0077-39e5-4d22-a21c-4854745002d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.862091 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f0077-39e5-4d22-a21c-4854745002d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.862938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f0077-39e5-4d22-a21c-4854745002d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.864703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f0077-39e5-4d22-a21c-4854745002d9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.867472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-42c383b0-13c3-4b1d-a576-d547587582a3 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:51.868706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-42c383b0-13c3-4b1d-a576-d547587582a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:51.875423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 796/1597] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:51.881127 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f0077-39e5-4d22-a21c-4854745002d9 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.881275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 802/1598] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:51.881275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c383b0-13c3-4b1d-a576-d547587582a3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.890746 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f0077-39e5-4d22-a21c-4854745002d9 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.892741 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c383b0-13c3-4b1d-a576-d547587582a3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.896329 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d1ec1-c610-4a1d-a578-2cea93184781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.896944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d1ec1-c610-4a1d-a578-2cea93184781 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.897235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d1ec1-c610-4a1d-a578-2cea93184781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.897510 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d1ec1-c610-4a1d-a578-2cea93184781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.898200 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d1ec1-c610-4a1d-a578-2cea93184781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.899828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d1ec1-c610-4a1d-a578-2cea93184781 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.899954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df774e5-ecb1-4250-bc81-461581e6bfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.900938 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df774e5-ecb1-4250-bc81-461581e6bfe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.901469 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df774e5-ecb1-4250-bc81-461581e6bfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.901957 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df774e5-ecb1-4250-bc81-461581e6bfe1 admin admin] PATH_INFO: `/v3/projects/5493cbcbaefb44f192bdffa1b3f6fd05/users/365d435bfe3a4c00956382364fb47edc/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.917101 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 797/1599] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:51.917101 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d1ec1-c610-4a1d-a578-2cea93184781 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:51.921389 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df774e5-ecb1-4250-bc81-461581e6bfe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5493cbcbaefb44f192bdffa1b3f6fd05, user_id=365d435bfe3a4c00956382364fb47edc, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.926852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df774e5-ecb1-4250-bc81-461581e6bfe1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:51.932621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d1ec1-c610-4a1d-a578-2cea93184781 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:51.941183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6af082-268e-4723-8620-fcea4508f5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:51.942388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6af082-268e-4723-8620-fcea4508f5ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:51.942962 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6af082-268e-4723-8620-fcea4508f5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:51.943498 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6af082-268e-4723-8620-fcea4508f5ef admin admin] PATH_INFO: `/v3/projects/933ef2ab34a441b982b69839434d689e/users/5d6b6321705947b6a395f77c11b1c6a5/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:51.969750 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6af082-268e-4723-8620-fcea4508f5ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=933ef2ab34a441b982b69839434d689e, user_id=5d6b6321705947b6a395f77c11b1c6a5, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:51.970901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-2e6af082-268e-4723-8620-fcea4508f5ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 12:05:51.970901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Sep 14 12:05:51.970901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: Sep 14 12:05:51.973469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6af082-268e-4723-8620-fcea4508f5ef admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:52.039726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 803/1600] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:05:51 2022] PUT /identity/v3/projects/5493cbcbaefb44f192bdffa1b3f6fd05/users/365d435bfe3a4c00956382364fb47edc/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:05:52.040591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df774e5-ecb1-4250-bc81-461581e6bfe1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:52.082269 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:52.095348 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 798/1601] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:05:51 2022] PUT /identity/v3/projects/933ef2ab34a441b982b69839434d689e/users/5d6b6321705947b6a395f77c11b1c6a5/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:05:52.098887 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6af082-268e-4723-8620-fcea4508f5ef admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:52.110006 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df774e5-ecb1-4250-bc81-461581e6bfe1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:52.188953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6af082-268e-4723-8620-fcea4508f5ef admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:52.188953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46aaf4-9816-474f-a743-0b816d8af64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:52.188953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46aaf4-9816-474f-a743-0b816d8af64d admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:52.188953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46aaf4-9816-474f-a743-0b816d8af64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:52.190569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46aaf4-9816-474f-a743-0b816d8af64d admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:52.190569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46aaf4-9816-474f-a743-0b816d8af64d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:52.190569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46aaf4-9816-474f-a743-0b816d8af64d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:52.203285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0962569-1679-4498-9f55-440fab86fa55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:52.204433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0962569-1679-4498-9f55-440fab86fa55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:52.205601 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0962569-1679-4498-9f55-440fab86fa55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:52.206000 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0962569-1679-4498-9f55-440fab86fa55 admin admin] PATH_INFO: `/v3/projects/748b7c08460b4f718dfff67275d8e4e4/users/4bd4125ff6bb400bae1b94df7d2cfa17/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:52.215218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 799/1602] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:52.224011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46aaf4-9816-474f-a743-0b816d8af64d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:52.229891 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0962569-1679-4498-9f55-440fab86fa55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=748b7c08460b4f718dfff67275d8e4e4, user_id=4bd4125ff6bb400bae1b94df7d2cfa17, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:52.233474 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0962569-1679-4498-9f55-440fab86fa55 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:52.233700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46aaf4-9816-474f-a743-0b816d8af64d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:52.239120 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd3c63-6ddd-4e0c-8bf9-dbb2477dee7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:52.240247 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd3c63-6ddd-4e0c-8bf9-dbb2477dee7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:52.240737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd3c63-6ddd-4e0c-8bf9-dbb2477dee7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:52.241097 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd3c63-6ddd-4e0c-8bf9-dbb2477dee7b admin admin] PATH_INFO: `/v3/projects/c1187680ceae42e28656e71f18234948/users/2f8bdae4e76147409ae32006ead1de80/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:52.245136 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:52.258724 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd3c63-6ddd-4e0c-8bf9-dbb2477dee7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1187680ceae42e28656e71f18234948, user_id=2f8bdae4e76147409ae32006ead1de80, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:52.261436 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd3c63-6ddd-4e0c-8bf9-dbb2477dee7b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:52.305509 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 804/1603] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:05:52 2022] PUT /identity/v3/projects/748b7c08460b4f718dfff67275d8e4e4/users/4bd4125ff6bb400bae1b94df7d2cfa17/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:05:52.306422 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0962569-1679-4498-9f55-440fab86fa55 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:52.334956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 800/1604] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:05:52 2022] PUT /identity/v3/projects/c1187680ceae42e28656e71f18234948/users/2f8bdae4e76147409ae32006ead1de80/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:05:52.335562 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd3c63-6ddd-4e0c-8bf9-dbb2477dee7b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:52.375749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0962569-1679-4498-9f55-440fab86fa55 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:52.413034 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd3c63-6ddd-4e0c-8bf9-dbb2477dee7b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:52.420851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc9fc3-9c0c-4a03-8990-22696509a1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:52.422062 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc9fc3-9c0c-4a03-8990-22696509a1ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:52.422567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc9fc3-9c0c-4a03-8990-22696509a1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:52.423070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc9fc3-9c0c-4a03-8990-22696509a1ff admin admin] PATH_INFO: `/v3/projects/c93eb4d32fd643328949c7edeb3a360d/users/41195f33c5f74c9bbea21f5fc177e2e5/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:52.443218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc9fc3-9c0c-4a03-8990-22696509a1ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c93eb4d32fd643328949c7edeb3a360d, user_id=41195f33c5f74c9bbea21f5fc177e2e5, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:52.446021 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc9fc3-9c0c-4a03-8990-22696509a1ff admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:52.454066 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb7323-7bd1-4201-bbf6-54bb69c90556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:52.454815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb7323-7bd1-4201-bbf6-54bb69c90556 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:52.455384 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb7323-7bd1-4201-bbf6-54bb69c90556 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:52.455903 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb7323-7bd1-4201-bbf6-54bb69c90556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:52.468232 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb7323-7bd1-4201-bbf6-54bb69c90556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:52.470112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb7323-7bd1-4201-bbf6-54bb69c90556 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:52.483472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 805/1605] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:05:52.484969 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e9974-3699-4ebd-8fb6-57f3e17abb6f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:52.485353 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e9974-3699-4ebd-8fb6-57f3e17abb6f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:52.485931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e9974-3699-4ebd-8fb6-57f3e17abb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:52.507564 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4e1e9974-3699-4ebd-8fb6-57f3e17abb6f None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:52.508636 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4e1e9974-3699-4ebd-8fb6-57f3e17abb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:52.523770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 806/1606] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:52.527674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 801/1607] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:05:52 2022] PUT /identity/v3/projects/c93eb4d32fd643328949c7edeb3a360d/users/41195f33c5f74c9bbea21f5fc177e2e5/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:05:52.531264 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc9fc3-9c0c-4a03-8990-22696509a1ff admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:52.539473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661d310-1f34-4bd3-a40d-8d959cdd2f74 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:52.539880 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661d310-1f34-4bd3-a40d-8d959cdd2f74 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:52.542927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661d310-1f34-4bd3-a40d-8d959cdd2f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:52.560838 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7661d310-1f34-4bd3-a40d-8d959cdd2f74 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:52.562080 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7661d310-1f34-4bd3-a40d-8d959cdd2f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:52.576505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:52.635381 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc9fc3-9c0c-4a03-8990-22696509a1ff admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:52.640468 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7c82b-854f-49ff-9a56-09ed896ea954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:52.641696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7c82b-854f-49ff-9a56-09ed896ea954 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:52.641696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7c82b-854f-49ff-9a56-09ed896ea954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:52.646137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7c82b-854f-49ff-9a56-09ed896ea954 admin admin] PATH_INFO: `/v3/projects/efcd7551351e41f7895b38ca6ad7271b/users/91ca8b5941a84fa580672dea3d78a9b9/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:52.654678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 807/1608] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:52.657688 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8552b6-68db-4a4a-959e-19a890213f3a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:52.657688 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8552b6-68db-4a4a-959e-19a890213f3a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:52.658344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8552b6-68db-4a4a-959e-19a890213f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:52.660737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7c82b-854f-49ff-9a56-09ed896ea954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efcd7551351e41f7895b38ca6ad7271b, user_id=91ca8b5941a84fa580672dea3d78a9b9, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:52.663819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7c82b-854f-49ff-9a56-09ed896ea954 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:52.682698 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4f8552b6-68db-4a4a-959e-19a890213f3a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:52.686463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4f8552b6-68db-4a4a-959e-19a890213f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:52.696219 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 808/1609] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:52.696219 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae93aad-bb76-4054-83b4-75b243025bcb None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:52.696219 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae93aad-bb76-4054-83b4-75b243025bcb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:52.696219 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae93aad-bb76-4054-83b4-75b243025bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:52.732336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-cae93aad-bb76-4054-83b4-75b243025bcb None None] MFA Rules not processed for user `2f8bdae4e76147409ae32006ead1de80`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:52.733697 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cae93aad-bb76-4054-83b4-75b243025bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:52.747983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 802/1610] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:05:52 2022] PUT /identity/v3/projects/efcd7551351e41f7895b38ca6ad7271b/users/91ca8b5941a84fa580672dea3d78a9b9/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:05:52.749374 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7c82b-854f-49ff-9a56-09ed896ea954 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:52.763157 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-aab7c82b-854f-49ff-9a56-09ed896ea954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:52.844788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 809/1611] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:52.846614 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cc875-40e3-43a0-a16e-ebcd461b20ab None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:52.847043 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cc875-40e3-43a0-a16e-ebcd461b20ab None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:52.847463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cc875-40e3-43a0-a16e-ebcd461b20ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:52.861697 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7c82b-854f-49ff-9a56-09ed896ea954 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:52.868501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2f923-9c56-410d-a473-017540f1d753 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:52.869630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2f923-9c56-410d-a473-017540f1d753 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:52.870187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2f923-9c56-410d-a473-017540f1d753 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:52.870743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2f923-9c56-410d-a473-017540f1d753 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:52.883084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1a0cc875-40e3-43a0-a16e-ebcd461b20ab None None] MFA Rules not processed for user `41195f33c5f74c9bbea21f5fc177e2e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:52.883822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:52.885515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1a0cc875-40e3-43a0-a16e-ebcd461b20ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:52.967241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d2f923-9c56-410d-a473-017540f1d753 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:52.968804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d2f923-9c56-410d-a473-017540f1d753 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:52.981269 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 803/1612] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:05:52.981979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2f923-9c56-410d-a473-017540f1d753 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:52.992373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2f923-9c56-410d-a473-017540f1d753 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:52.997105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7bd1d-5f16-441c-9855-83fec7e90f58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:52.997776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7bd1d-5f16-441c-9855-83fec7e90f58 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:52.998261 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7bd1d-5f16-441c-9855-83fec7e90f58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:52.998716 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7bd1d-5f16-441c-9855-83fec7e90f58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:53.011886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7bd1d-5f16-441c-9855-83fec7e90f58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:53.014469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7bd1d-5f16-441c-9855-83fec7e90f58 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:53.026678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 810/1613] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:53.028165 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 804/1614] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:05:53.029658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13a992-17c9-4167-85b9-8ddf59571114 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:53.030425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13a992-17c9-4167-85b9-8ddf59571114 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:53.030738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13a992-17c9-4167-85b9-8ddf59571114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:53.032245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cc875-40e3-43a0-a16e-ebcd461b20ab None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:53.067177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cc875-40e3-43a0-a16e-ebcd461b20ab None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:53.073254 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-bd13a992-17c9-4167-85b9-8ddf59571114 None None] MFA Rules not processed for user `91ca8b5941a84fa580672dea3d78a9b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:53.073254 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-bd13a992-17c9-4167-85b9-8ddf59571114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:53.079399 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f6de5-b59e-4de3-9aa4-122d264f1522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:53.080484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f6de5-b59e-4de3-9aa4-122d264f1522 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:53.080976 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f6de5-b59e-4de3-9aa4-122d264f1522 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:53.082100 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f6de5-b59e-4de3-9aa4-122d264f1522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:53.099874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f6de5-b59e-4de3-9aa4-122d264f1522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:53.102811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f6de5-b59e-4de3-9aa4-122d264f1522 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:53.122825 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 811/1615] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:05:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:05:53.124600 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f6de5-b59e-4de3-9aa4-122d264f1522 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:53.142118 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f6de5-b59e-4de3-9aa4-122d264f1522 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:53.152834 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a6669-176c-48f2-9f47-13e9c7dae649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:53.154721 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a6669-176c-48f2-9f47-13e9c7dae649 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:53.155002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a6669-176c-48f2-9f47-13e9c7dae649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:53.160470 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a6669-176c-48f2-9f47-13e9c7dae649 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:53.160470 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a6669-176c-48f2-9f47-13e9c7dae649 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:53.162955 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a6669-176c-48f2-9f47-13e9c7dae649 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:53.208928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:53.221990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 805/1616] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:53.224485 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431ce03-4785-4e4a-a46d-fe670593d54d None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:53.229051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431ce03-4785-4e4a-a46d-fe670593d54d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:53.229051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431ce03-4785-4e4a-a46d-fe670593d54d None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:53.238362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 806/1617] 10.222.0.62 () {54 vars in 962 bytes} [Wed Sep 14 12:05:53 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:05:53.241111 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc697d9a-c276-467e-b505-6573b3733b3c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:53.241731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc697d9a-c276-467e-b505-6573b3733b3c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:53.242360 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc697d9a-c276-467e-b505-6573b3733b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:53.308036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 812/1618] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:05:53 2022] POST /identity/v3/projects => generated 373 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:53.314159 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f58ce-d3fb-4711-b1a8-ac10fb78f972 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:53.315857 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f58ce-d3fb-4711-b1a8-ac10fb78f972 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:53.316879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f58ce-d3fb-4711-b1a8-ac10fb78f972 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:53.321029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 813/1619] 10.222.0.62 () {54 vars in 962 bytes} [Wed Sep 14 12:05:53 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:05:53.322720 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f58ce-d3fb-4711-b1a8-ac10fb78f972 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:53.328194 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-bc697d9a-c276-467e-b505-6573b3733b3c None None] MFA Rules not processed for user `38b093bc4c1d4ebcbd2a6bb16b70c029`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:53.334929 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-bc697d9a-c276-467e-b505-6573b3733b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:53.456101 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:53.472557 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f58ce-d3fb-4711-b1a8-ac10fb78f972 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:53.479813 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528b6a5-181b-4f26-a85c-da4118a32080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:53.480659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528b6a5-181b-4f26-a85c-da4118a32080 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:53.486620 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528b6a5-181b-4f26-a85c-da4118a32080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:53.486620 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528b6a5-181b-4f26-a85c-da4118a32080 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:53.486620 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528b6a5-181b-4f26-a85c-da4118a32080 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:53.486620 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528b6a5-181b-4f26-a85c-da4118a32080 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:53.492970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 807/1620] 10.222.0.62 () {58 vars in 1059 bytes} [Wed Sep 14 12:05:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:53.495611 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a26ac6-cce3-4e65-a07c-0643d6bbfa10 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:53.496116 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a26ac6-cce3-4e65-a07c-0643d6bbfa10 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:53.496609 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a26ac6-cce3-4e65-a07c-0643d6bbfa10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:53.536650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-04a26ac6-cce3-4e65-a07c-0643d6bbfa10 None None] MFA Rules not processed for user `38b093bc4c1d4ebcbd2a6bb16b70c029`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:53.539204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-04a26ac6-cce3-4e65-a07c-0643d6bbfa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:53.558982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 808/1621] 10.222.0.62 () {58 vars in 1059 bytes} [Wed Sep 14 12:05:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:53.560613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a26ac6-cce3-4e65-a07c-0643d6bbfa10 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:53.583898 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a26ac6-cce3-4e65-a07c-0643d6bbfa10 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:53.602466 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18dc50d-6018-49b6-9bf4-168b6757ad5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:53.603430 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dc50d-6018-49b6-9bf4-168b6757ad5d service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:53.603931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dc50d-6018-49b6-9bf4-168b6757ad5d service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:53.604413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dc50d-6018-49b6-9bf4-168b6757ad5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:53.646534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 814/1622] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:05:53 2022] POST /identity/v3/users => generated 293 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:53.648192 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528b6a5-181b-4f26-a85c-da4118a32080 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:53.665825 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528b6a5-181b-4f26-a85c-da4118a32080 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:53.672972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e65a4c-5061-4715-834f-79ab7d86a224 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:53.676790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e65a4c-5061-4715-834f-79ab7d86a224 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:53.676790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e65a4c-5061-4715-834f-79ab7d86a224 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:53.676790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e65a4c-5061-4715-834f-79ab7d86a224 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:53.814745 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18dc50d-6018-49b6-9bf4-168b6757ad5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:53.821120 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18dc50d-6018-49b6-9bf4-168b6757ad5d service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:53.821828 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e65a4c-5061-4715-834f-79ab7d86a224 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:53.824988 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e65a4c-5061-4715-834f-79ab7d86a224 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:53.852245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 809/1623] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:05:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:05:53.854356 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18dc50d-6018-49b6-9bf4-168b6757ad5d service nova] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:53.874659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 815/1624] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:05:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:05:53.939372 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18dc50d-6018-49b6-9bf4-168b6757ad5d service nova] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:53.974325 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52e4fd-e538-4b04-be13-5fd0a18c91a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:53.981578 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52e4fd-e538-4b04-be13-5fd0a18c91a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:53.983602 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52e4fd-e538-4b04-be13-5fd0a18c91a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:53.984379 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52e4fd-e538-4b04-be13-5fd0a18c91a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:54.000978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52e4fd-e538-4b04-be13-5fd0a18c91a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:54.014435 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52e4fd-e538-4b04-be13-5fd0a18c91a6 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:54.082621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 810/1625] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:54.085324 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e65a4c-5061-4715-834f-79ab7d86a224 service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:54.100803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e65a4c-5061-4715-834f-79ab7d86a224 service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:54.112039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b12a6-fd32-479c-89ef-c03477c78ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:54.115419 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b12a6-fd32-479c-89ef-c03477c78ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:54.115711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b12a6-fd32-479c-89ef-c03477c78ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:54.115991 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b12a6-fd32-479c-89ef-c03477c78ed8 admin admin] PATH_INFO: `/v3/projects/14ea26a132e844ec9d7c02540909e8cf/users/aba5dc96d6994997ab1df353053e15d9/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:54.176416 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b12a6-fd32-479c-89ef-c03477c78ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14ea26a132e844ec9d7c02540909e8cf, user_id=aba5dc96d6994997ab1df353053e15d9, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:54.197728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b12a6-fd32-479c-89ef-c03477c78ed8 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:54.341142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 816/1626] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:05:54 2022] PUT /identity/v3/projects/14ea26a132e844ec9d7c02540909e8cf/users/aba5dc96d6994997ab1df353053e15d9/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:05:54.345051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e132d8-752c-4423-89d9-9470ace1674b None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:54.345815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e132d8-752c-4423-89d9-9470ace1674b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:54.345815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e132d8-752c-4423-89d9-9470ace1674b None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:54.347074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 811/1627] 10.222.0.62 () {54 vars in 912 bytes} [Wed Sep 14 12:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 12:05:54.348129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b12a6-fd32-479c-89ef-c03477c78ed8 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:54.373489 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93adf31-7450-4a95-be34-9dfda9b9ddcd None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:54.373489 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93adf31-7450-4a95-be34-9dfda9b9ddcd None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:54.373489 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93adf31-7450-4a95-be34-9dfda9b9ddcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:54.440864 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d93adf31-7450-4a95-be34-9dfda9b9ddcd None None] MFA Rules not processed for user `38b093bc4c1d4ebcbd2a6bb16b70c029`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:54.444879 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d93adf31-7450-4a95-be34-9dfda9b9ddcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:54.508083 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b12a6-fd32-479c-89ef-c03477c78ed8 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:54.520364 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d2fb8-1089-4690-abf6-e8ed5c70cc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:54.716772 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d2fb8-1089-4690-abf6-e8ed5c70cc22 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:54.716772 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d2fb8-1089-4690-abf6-e8ed5c70cc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:54.716772 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d2fb8-1089-4690-abf6-e8ed5c70cc22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:54.716772 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d2fb8-1089-4690-abf6-e8ed5c70cc22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:54.716772 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d2fb8-1089-4690-abf6-e8ed5c70cc22 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:54.716772 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:54.716772 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 817/1628] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:54.719625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d2fb8-1089-4690-abf6-e8ed5c70cc22 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:54.719625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d2fb8-1089-4690-abf6-e8ed5c70cc22 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:54.719625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f3f79-085a-4c15-8743-6ebc3728b4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:54.719625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f3f79-085a-4c15-8743-6ebc3728b4de admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:54.720623 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f3f79-085a-4c15-8743-6ebc3728b4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:54.720623 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f3f79-085a-4c15-8743-6ebc3728b4de admin admin] PATH_INFO: `/v3/projects/14ea26a132e844ec9d7c02540909e8cf/users/aba5dc96d6994997ab1df353053e15d9/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:54.720623 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f3f79-085a-4c15-8743-6ebc3728b4de admin admin] RBAC: Authorizing `identity:create_grant(project_id=14ea26a132e844ec9d7c02540909e8cf, user_id=aba5dc96d6994997ab1df353053e15d9, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:54.720623 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f3f79-085a-4c15-8743-6ebc3728b4de admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:54.720623 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 818/1629] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:05:54 2022] PUT /identity/v3/projects/14ea26a132e844ec9d7c02540909e8cf/users/aba5dc96d6994997ab1df353053e15d9/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:05:54.721295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 812/1630] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 14 12:05:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:05:54.721777 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2c70e-7335-48cf-89c0-bf603b411a2a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:54.721777 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2c70e-7335-48cf-89c0-bf603b411a2a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:54.721777 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2c70e-7335-48cf-89c0-bf603b411a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:54.722789 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93adf31-7450-4a95-be34-9dfda9b9ddcd None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:54.774536 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-83a2c70e-7335-48cf-89c0-bf603b411a2a None None] MFA Rules not processed for user `aba5dc96d6994997ab1df353053e15d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:54.776399 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-83a2c70e-7335-48cf-89c0-bf603b411a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:54.789696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93adf31-7450-4a95-be34-9dfda9b9ddcd None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:54.856804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:54.926610 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 819/1631] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:54.927674 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796c267-28af-428f-802b-ebee14970c12 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:54.928155 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796c267-28af-428f-802b-ebee14970c12 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:54.937897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796c267-28af-428f-802b-ebee14970c12 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:54.939409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69386140-ba27-42a0-98e7-07d8ac3df499 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:54.940461 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 820/1632] 10.222.0.62 () {54 vars in 912 bytes} [Wed Sep 14 12:05:54 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:05:54.940696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69386140-ba27-42a0-98e7-07d8ac3df499 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:54.943425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69386140-ba27-42a0-98e7-07d8ac3df499 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:54.943425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69386140-ba27-42a0-98e7-07d8ac3df499 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:54.948432 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796c267-28af-428f-802b-ebee14970c12 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:54.967987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796c267-28af-428f-802b-ebee14970c12 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:54.975474 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2d863-7c2a-40bf-b77c-d7916fb22b07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:54.976778 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2d863-7c2a-40bf-b77c-d7916fb22b07 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:54.977384 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2d863-7c2a-40bf-b77c-d7916fb22b07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:54.978044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2d863-7c2a-40bf-b77c-d7916fb22b07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:54.995557 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2d863-7c2a-40bf-b77c-d7916fb22b07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:54.997601 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2d863-7c2a-40bf-b77c-d7916fb22b07 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:55.020125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 821/1633] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:05:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:05:55.022010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702423ab-042d-41cf-8601-aefae4572239 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:55.022868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702423ab-042d-41cf-8601-aefae4572239 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:55.023338 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702423ab-042d-41cf-8601-aefae4572239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:55.061144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-702423ab-042d-41cf-8601-aefae4572239 None None] MFA Rules not processed for user `38b093bc4c1d4ebcbd2a6bb16b70c029`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:55.064428 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-702423ab-042d-41cf-8601-aefae4572239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:55.207430 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69386140-ba27-42a0-98e7-07d8ac3df499 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:55.213714 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69386140-ba27-42a0-98e7-07d8ac3df499 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:55.227953 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 822/1634] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 14 12:05:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:55.280494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 813/1635] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:05:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:05:55.304073 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702423ab-042d-41cf-8601-aefae4572239 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:55.337236 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702423ab-042d-41cf-8601-aefae4572239 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:55.349598 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c993497-d125-4190-ba1c-c9ca0391888d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:55.350806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c993497-d125-4190-ba1c-c9ca0391888d service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:55.351527 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c993497-d125-4190-ba1c-c9ca0391888d service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:55.352018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c993497-d125-4190-ba1c-c9ca0391888d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:55.380720 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c993497-d125-4190-ba1c-c9ca0391888d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:55.383852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c993497-d125-4190-ba1c-c9ca0391888d service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:55.420505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 823/1636] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:05:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:05:55.964545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69386140-ba27-42a0-98e7-07d8ac3df499 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:56.156492 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69386140-ba27-42a0-98e7-07d8ac3df499 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:56.168828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6aae8d-8388-4b7b-babb-0982a5235f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:56.175942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6aae8d-8388-4b7b-babb-0982a5235f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:56.176310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6aae8d-8388-4b7b-babb-0982a5235f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:56.176683 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6aae8d-8388-4b7b-babb-0982a5235f7a admin admin] PATH_INFO: `/v3/users/41195f33c5f74c9bbea21f5fc177e2e5` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:56.179358 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6aae8d-8388-4b7b-babb-0982a5235f7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=41195f33c5f74c9bbea21f5fc177e2e5)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:56.181846 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-6a6aae8d-8388-4b7b-babb-0982a5235f7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 12:05:56.181846 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Sep 14 12:05:56.181846 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: Sep 14 12:05:56.191615 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6aae8d-8388-4b7b-babb-0982a5235f7a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:56.476475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd670a1-a315-40c6-b8b6-a290f08e3c47 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:56.477431 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd670a1-a315-40c6-b8b6-a290f08e3c47 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:56.477913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd670a1-a315-40c6-b8b6-a290f08e3c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:56.499515 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6a6aae8d-8388-4b7b-babb-0982a5235f7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41195f33c5f74c9bbea21f5fc177e2e5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:05:56.544724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-cdd670a1-a315-40c6-b8b6-a290f08e3c47 None None] MFA Rules not processed for user `365d435bfe3a4c00956382364fb47edc`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:56.546536 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cdd670a1-a315-40c6-b8b6-a290f08e3c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:56.561999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6a6aae8d-8388-4b7b-babb-0982a5235f7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41195f33c5f74c9bbea21f5fc177e2e5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:05:56.575909 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:56.583398 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:56.608501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6a6aae8d-8388-4b7b-babb-0982a5235f7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41195f33c5f74c9bbea21f5fc177e2e5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:05:56.671270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 824/1637] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:56 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:56.673292 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd670a1-a315-40c6-b8b6-a290f08e3c47 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:56.686996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd670a1-a315-40c6-b8b6-a290f08e3c47 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:56.693530 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21459eb3-0ecc-4ce6-a924-85908032a134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:56.695180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21459eb3-0ecc-4ce6-a924-85908032a134 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:56.695180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21459eb3-0ecc-4ce6-a924-85908032a134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:56.695710 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21459eb3-0ecc-4ce6-a924-85908032a134 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:56.696454 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21459eb3-0ecc-4ce6-a924-85908032a134 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:56.698752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21459eb3-0ecc-4ce6-a924-85908032a134 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:56.775770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6a6aae8d-8388-4b7b-babb-0982a5235f7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41195f33c5f74c9bbea21f5fc177e2e5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:05:56.776547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6a6aae8d-8388-4b7b-babb-0982a5235f7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41195f33c5f74c9bbea21f5fc177e2e5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:05:56.799604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 814/1638] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:05:55 2022] DELETE /identity/v3/users/41195f33c5f74c9bbea21f5fc177e2e5 => generated 0 bytes in 830 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:05:56.853765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10428da9-4bf7-4f5c-ad87-d9560d95fe0b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:56.854805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 825/1639] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:05:56 2022] POST /identity/v3/projects => generated 405 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:56.856164 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21459eb3-0ecc-4ce6-a924-85908032a134 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:56.857001 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10428da9-4bf7-4f5c-ad87-d9560d95fe0b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:56.857470 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10428da9-4bf7-4f5c-ad87-d9560d95fe0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:56.858808 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-21459eb3-0ecc-4ce6-a924-85908032a134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:56.992645 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-10428da9-4bf7-4f5c-ad87-d9560d95fe0b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:56.997671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-10428da9-4bf7-4f5c-ad87-d9560d95fe0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:57.087418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21459eb3-0ecc-4ce6-a924-85908032a134 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:57.097443 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1435210-0aa6-4668-bb48-809e8362be25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:57.098144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1435210-0aa6-4668-bb48-809e8362be25 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:57.099403 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1435210-0aa6-4668-bb48-809e8362be25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:57.099883 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1435210-0aa6-4668-bb48-809e8362be25 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:57.103181 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1435210-0aa6-4668-bb48-809e8362be25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:57.107307 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1435210-0aa6-4668-bb48-809e8362be25 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:57.174162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 815/1640] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:57.176688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10428da9-4bf7-4f5c-ad87-d9560d95fe0b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:57.178430 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-10428da9-4bf7-4f5c-ad87-d9560d95fe0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:57.198120 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10428da9-4bf7-4f5c-ad87-d9560d95fe0b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:57.217118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccdc8d6-48e1-4e30-b5d1-7a8c2c27c284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:57.218522 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccdc8d6-48e1-4e30-b5d1-7a8c2c27c284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:57.220448 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccdc8d6-48e1-4e30-b5d1-7a8c2c27c284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:57.220844 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccdc8d6-48e1-4e30-b5d1-7a8c2c27c284 admin admin] PATH_INFO: `/v3/users/2f8bdae4e76147409ae32006ead1de80` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:57.223094 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccdc8d6-48e1-4e30-b5d1-7a8c2c27c284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f8bdae4e76147409ae32006ead1de80)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:57.235994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccdc8d6-48e1-4e30-b5d1-7a8c2c27c284 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:57.236133 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 826/1641] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:05:56 2022] POST /identity/v3/users => generated 309 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:05:57.240104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1435210-0aa6-4668-bb48-809e8362be25 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:57.264963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1435210-0aa6-4668-bb48-809e8362be25 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:57.280906 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa63553-6d63-44c8-9d23-b9be6ca14805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:57.281787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa63553-6d63-44c8-9d23-b9be6ca14805 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:57.282227 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa63553-6d63-44c8-9d23-b9be6ca14805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:57.282651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa63553-6d63-44c8-9d23-b9be6ca14805 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:57.283580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa63553-6d63-44c8-9d23-b9be6ca14805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:57.285376 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa63553-6d63-44c8-9d23-b9be6ca14805 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:57.314427 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 827/1642] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:05:57.316302 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa63553-6d63-44c8-9d23-b9be6ca14805 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:57.321091 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-bfa63553-6d63-44c8-9d23-b9be6ca14805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:57.451606 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa63553-6d63-44c8-9d23-b9be6ca14805 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:57.462162 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af1bb9-871b-4841-b6c8-02e9b23e9215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:57.464090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af1bb9-871b-4841-b6c8-02e9b23e9215 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:57.464090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af1bb9-871b-4841-b6c8-02e9b23e9215 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:57.464874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af1bb9-871b-4841-b6c8-02e9b23e9215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:57.478347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af1bb9-871b-4841-b6c8-02e9b23e9215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:57.479922 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af1bb9-871b-4841-b6c8-02e9b23e9215 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:57.497357 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 828/1643] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:05:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:05:57.506703 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af1bb9-871b-4841-b6c8-02e9b23e9215 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:57.530443 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af1bb9-871b-4841-b6c8-02e9b23e9215 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:57.542726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f018d4e-e1ba-442d-bd9c-97486f621ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:57.543417 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f018d4e-e1ba-442d-bd9c-97486f621ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:57.543878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f018d4e-e1ba-442d-bd9c-97486f621ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:57.544795 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f018d4e-e1ba-442d-bd9c-97486f621ed2 admin admin] PATH_INFO: `/v3/projects/660c8ff838b2437093a7823701be08ec/users/d9d8955f8bdb499eab93afb503505ccd/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:57.576505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f018d4e-e1ba-442d-bd9c-97486f621ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=660c8ff838b2437093a7823701be08ec, user_id=d9d8955f8bdb499eab93afb503505ccd, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:57.579487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f018d4e-e1ba-442d-bd9c-97486f621ed2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:57.708188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4ccdc8d6-48e1-4e30-b5d1-7a8c2c27c284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f8bdae4e76147409ae32006ead1de80'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:05:57.713190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 829/1644] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:05:57 2022] PUT /identity/v3/projects/660c8ff838b2437093a7823701be08ec/users/d9d8955f8bdb499eab93afb503505ccd/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:05:57.714652 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f018d4e-e1ba-442d-bd9c-97486f621ed2 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:57.740265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4ccdc8d6-48e1-4e30-b5d1-7a8c2c27c284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f8bdae4e76147409ae32006ead1de80'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:05:57.754637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4ccdc8d6-48e1-4e30-b5d1-7a8c2c27c284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f8bdae4e76147409ae32006ead1de80'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:05:57.802239 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:57.854827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4ccdc8d6-48e1-4e30-b5d1-7a8c2c27c284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f8bdae4e76147409ae32006ead1de80'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:05:57.858270 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4ccdc8d6-48e1-4e30-b5d1-7a8c2c27c284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f8bdae4e76147409ae32006ead1de80'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:05:57.867186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 816/1645] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:05:57 2022] DELETE /identity/v3/users/2f8bdae4e76147409ae32006ead1de80 => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:05:57.870342 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f018d4e-e1ba-442d-bd9c-97486f621ed2 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:57.871974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1314f-7d26-4640-9b04-5c95aacd1b0a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:57.872268 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1314f-7d26-4640-9b04-5c95aacd1b0a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:57.872549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1314f-7d26-4640-9b04-5c95aacd1b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:57.913254 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-90c1314f-7d26-4640-9b04-5c95aacd1b0a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:57.914562 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-90c1314f-7d26-4640-9b04-5c95aacd1b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:57.998776 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe978b-b58f-4bf5-814c-6efece9023f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:57.999419 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe978b-b58f-4bf5-814c-6efece9023f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:57.999697 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe978b-b58f-4bf5-814c-6efece9023f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:58.000146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe978b-b58f-4bf5-814c-6efece9023f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:58.000643 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe978b-b58f-4bf5-814c-6efece9023f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:58.002346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe978b-b58f-4bf5-814c-6efece9023f2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:58.026811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 830/1646] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:05:58.036107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe978b-b58f-4bf5-814c-6efece9023f2 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:58.036107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a9fe978b-b58f-4bf5-814c-6efece9023f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:58.041251 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:05:58.052577 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe978b-b58f-4bf5-814c-6efece9023f2 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:58.061075 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 817/1647] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:05:58.063990 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7185787-a64a-4b13-854a-a81a72784cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:58.063990 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7185787-a64a-4b13-854a-a81a72784cef admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:58.063990 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7185787-a64a-4b13-854a-a81a72784cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:58.063990 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7185787-a64a-4b13-854a-a81a72784cef admin admin] PATH_INFO: `/v3/projects/660c8ff838b2437093a7823701be08ec/users/d9d8955f8bdb499eab93afb503505ccd/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:58.066703 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7185787-a64a-4b13-854a-a81a72784cef admin admin] RBAC: Authorizing `identity:create_grant(project_id=660c8ff838b2437093a7823701be08ec, user_id=d9d8955f8bdb499eab93afb503505ccd, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:58.070950 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7185787-a64a-4b13-854a-a81a72784cef admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:58.086552 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 831/1648] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:05:58 2022] PUT /identity/v3/projects/660c8ff838b2437093a7823701be08ec/users/d9d8955f8bdb499eab93afb503505ccd/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:05:58.115358 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1314f-7d26-4640-9b04-5c95aacd1b0a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:58.117913 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-90c1314f-7d26-4640-9b04-5c95aacd1b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:58.240605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1314f-7d26-4640-9b04-5c95aacd1b0a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:58.325926 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00871687-98bf-40ed-badd-7db35b637111 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:58.327775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00871687-98bf-40ed-badd-7db35b637111 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:58.329247 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00871687-98bf-40ed-badd-7db35b637111 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:58.330190 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00871687-98bf-40ed-badd-7db35b637111 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:58.430742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00871687-98bf-40ed-badd-7db35b637111 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:58.434246 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00871687-98bf-40ed-badd-7db35b637111 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:58.453253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 818/1649] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:05:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:05:59.471524 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7185787-a64a-4b13-854a-a81a72784cef admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:05:59.476960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a7185787-a64a-4b13-854a-a81a72784cef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:05:59.498866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7185787-a64a-4b13-854a-a81a72784cef admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:05:59.505314 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cade2-0a2d-45fc-b8bd-e991b971f0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:05:59.506259 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cade2-0a2d-45fc-b8bd-e991b971f0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:59.506768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cade2-0a2d-45fc-b8bd-e991b971f0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:59.507247 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cade2-0a2d-45fc-b8bd-e991b971f0db admin admin] PATH_INFO: `/v3/projects/c93eb4d32fd643328949c7edeb3a360d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:59.508969 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8cade2-0a2d-45fc-b8bd-e991b971f0db admin admin] RBAC: Authorizing `identity:delete_project(project_id=c93eb4d32fd643328949c7edeb3a360d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:05:59.510283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-0c8cade2-0a2d-45fc-b8bd-e991b971f0db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 12:05:59.510283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Sep 14 12:05:59.510283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: Sep 14 12:05:59.512246 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8cade2-0a2d-45fc-b8bd-e991b971f0db admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:05:59.682255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0c8cade2-0a2d-45fc-b8bd-e991b971f0db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c93eb4d32fd643328949c7edeb3a360d', 'request_id': 'req-0c8cade2-0a2d-45fc-b8bd-e991b971f0db'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:05:59.746405 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0c8cade2-0a2d-45fc-b8bd-e991b971f0db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c93eb4d32fd643328949c7edeb3a360d', 'request_id': 'req-0c8cade2-0a2d-45fc-b8bd-e991b971f0db'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:05:59.831807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0c8cade2-0a2d-45fc-b8bd-e991b971f0db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c93eb4d32fd643328949c7edeb3a360d', 'request_id': 'req-0c8cade2-0a2d-45fc-b8bd-e991b971f0db'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:05:59.869768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0c8cade2-0a2d-45fc-b8bd-e991b971f0db admin admin] The token cache is being invalidate because project c93eb4d32fd643328949c7edeb3a360d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:05:59.870977 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0c8cade2-0a2d-45fc-b8bd-e991b971f0db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:05:59.874890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0c8cade2-0a2d-45fc-b8bd-e991b971f0db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:05:59.890280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c64b23-3382-4179-8621-bba320f8877d None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:05:59.890895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c64b23-3382-4179-8621-bba320f8877d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:05:59.891073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c64b23-3382-4179-8621-bba320f8877d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:05:59.910683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 832/1650] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:05:59 2022] DELETE /identity/v3/projects/c93eb4d32fd643328949c7edeb3a360d => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:05:59.944309 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-64c64b23-3382-4179-8621-bba320f8877d None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:05:59.945840 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-64c64b23-3382-4179-8621-bba320f8877d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:00.059128 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 819/1651] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:00.068533 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86639050-1599-4a64-9c2b-201f291eb8a7 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:00.069431 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86639050-1599-4a64-9c2b-201f291eb8a7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:00.069957 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86639050-1599-4a64-9c2b-201f291eb8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:00.096448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-86639050-1599-4a64-9c2b-201f291eb8a7 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:00.098816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-86639050-1599-4a64-9c2b-201f291eb8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:00.107796 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 833/1652] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:00.115110 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c64b23-3382-4179-8621-bba320f8877d None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:00.120344 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:00.124333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c64b23-3382-4179-8621-bba320f8877d None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:00.130786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d928450-9d43-4316-8a63-ac61ace4191e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:00.131699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d928450-9d43-4316-8a63-ac61ace4191e admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:00.132493 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d928450-9d43-4316-8a63-ac61ace4191e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:00.132964 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d928450-9d43-4316-8a63-ac61ace4191e admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:00.133901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d928450-9d43-4316-8a63-ac61ace4191e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:06:00.135588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d928450-9d43-4316-8a63-ac61ace4191e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:00.138057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d928450-9d43-4316-8a63-ac61ace4191e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:00.152904 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 820/1653] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:06:00.157004 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86639050-1599-4a64-9c2b-201f291eb8a7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:00.166048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:00.170766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86639050-1599-4a64-9c2b-201f291eb8a7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:00.177131 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b8aef-57eb-449b-8a07-44317cd32063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:00.180052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b8aef-57eb-449b-8a07-44317cd32063 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:00.180564 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b8aef-57eb-449b-8a07-44317cd32063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:00.181385 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b8aef-57eb-449b-8a07-44317cd32063 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:00.182783 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b8aef-57eb-449b-8a07-44317cd32063 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:00.186922 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b8aef-57eb-449b-8a07-44317cd32063 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:00.348174 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 834/1654] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:06:00 2022] POST /identity/v3/projects => generated 365 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:06:00.350164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d928450-9d43-4316-8a63-ac61ace4191e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:00.365103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d928450-9d43-4316-8a63-ac61ace4191e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:00.473705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb181f-5bc1-41de-bb87-52e2680accd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:00.475190 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb181f-5bc1-41de-bb87-52e2680accd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:00.475852 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb181f-5bc1-41de-bb87-52e2680accd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:00.476372 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb181f-5bc1-41de-bb87-52e2680accd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:00.477323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb181f-5bc1-41de-bb87-52e2680accd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:00.479618 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb181f-5bc1-41de-bb87-52e2680accd5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:00.530813 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b8aef-57eb-449b-8a07-44317cd32063 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:00.533959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3a5b8aef-57eb-449b-8a07-44317cd32063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:00.553483 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b8aef-57eb-449b-8a07-44317cd32063 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:00.558743 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff26778-5857-40ad-ade1-8eb242696794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:00.559747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff26778-5857-40ad-ade1-8eb242696794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:00.560333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff26778-5857-40ad-ade1-8eb242696794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:00.560819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff26778-5857-40ad-ade1-8eb242696794 admin admin] PATH_INFO: `/v3/projects/c1187680ceae42e28656e71f18234948` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:00.563066 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff26778-5857-40ad-ade1-8eb242696794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1187680ceae42e28656e71f18234948)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:00.565263 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff26778-5857-40ad-ade1-8eb242696794 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:00.605271 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 821/1655] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:06:00 2022] POST /identity/v3/users => generated 283 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:06:00.610837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb181f-5bc1-41de-bb87-52e2680accd5 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:00.623419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb181f-5bc1-41de-bb87-52e2680accd5 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:00.630891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4f32e-8282-417d-bf87-728c116551a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:00.631846 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4f32e-8282-417d-bf87-728c116551a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:00.632446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4f32e-8282-417d-bf87-728c116551a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:00.632910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4f32e-8282-417d-bf87-728c116551a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:00.641051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e4f32e-8282-417d-bf87-728c116551a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:00.643014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e4f32e-8282-417d-bf87-728c116551a8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:00.663649 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 822/1656] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:06:00.667734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4f32e-8282-417d-bf87-728c116551a8 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:00.680170 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4f32e-8282-417d-bf87-728c116551a8 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:00.688657 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5e630-147e-42b8-92e6-56bcbc941173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:00.689552 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5e630-147e-42b8-92e6-56bcbc941173 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:00.690059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5e630-147e-42b8-92e6-56bcbc941173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:00.691262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5e630-147e-42b8-92e6-56bcbc941173 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:00.692544 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5e630-147e-42b8-92e6-56bcbc941173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:00.694654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5e630-147e-42b8-92e6-56bcbc941173 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:00.710229 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 823/1657] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:06:00.714159 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5e630-147e-42b8-92e6-56bcbc941173 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:00.751249 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5e630-147e-42b8-92e6-56bcbc941173 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:00.809553 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cff26778-5857-40ad-ade1-8eb242696794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1187680ceae42e28656e71f18234948', 'request_id': 'req-cff26778-5857-40ad-ade1-8eb242696794'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:00.895764 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda23f5-0920-42f2-b3cc-a672191fe7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:00.901295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda23f5-0920-42f2-b3cc-a672191fe7ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:00.901295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda23f5-0920-42f2-b3cc-a672191fe7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:00.901295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda23f5-0920-42f2-b3cc-a672191fe7ff admin admin] PATH_INFO: `/v3/projects/c9b0af5514c94e178bc37e1ea5053060/users/f04baf2739c0430e8514faa2c42de104/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:00.922512 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cff26778-5857-40ad-ade1-8eb242696794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1187680ceae42e28656e71f18234948', 'request_id': 'req-cff26778-5857-40ad-ade1-8eb242696794'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:00.928413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda23f5-0920-42f2-b3cc-a672191fe7ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b0af5514c94e178bc37e1ea5053060, user_id=f04baf2739c0430e8514faa2c42de104, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:00.936692 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda23f5-0920-42f2-b3cc-a672191fe7ff admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:00.936777 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cff26778-5857-40ad-ade1-8eb242696794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1187680ceae42e28656e71f18234948', 'request_id': 'req-cff26778-5857-40ad-ade1-8eb242696794'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:00.968167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cff26778-5857-40ad-ade1-8eb242696794 admin admin] The token cache is being invalidate because project c1187680ceae42e28656e71f18234948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:06:00.968553 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cff26778-5857-40ad-ade1-8eb242696794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:00.969226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cff26778-5857-40ad-ade1-8eb242696794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:00.970923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 835/1658] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:06:00 2022] DELETE /identity/v3/projects/c1187680ceae42e28656e71f18234948 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:06:00.977405 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff26778-5857-40ad-ade1-8eb242696794 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:00.982084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cff26778-5857-40ad-ade1-8eb242696794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:01.022595 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff26778-5857-40ad-ade1-8eb242696794 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:01.036221 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 824/1659] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:06:00 2022] PUT /identity/v3/projects/c9b0af5514c94e178bc37e1ea5053060/users/f04baf2739c0430e8514faa2c42de104/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:06:01.036740 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f806c0-f859-4969-9272-48bcaff5400f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:01.037453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f806c0-f859-4969-9272-48bcaff5400f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:01.037743 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f806c0-f859-4969-9272-48bcaff5400f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:01.038044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f806c0-f859-4969-9272-48bcaff5400f admin admin] PATH_INFO: `/v3/users/aba5dc96d6994997ab1df353053e15d9` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:01.038246 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda23f5-0920-42f2-b3cc-a672191fe7ff admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:01.042550 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-edda23f5-0920-42f2-b3cc-a672191fe7ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:01.043767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f806c0-f859-4969-9272-48bcaff5400f admin admin] RBAC: Authorizing `identity:delete_user(user_id=aba5dc96d6994997ab1df353053e15d9)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:01.063139 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f806c0-f859-4969-9272-48bcaff5400f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:01.208394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:01.264282 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda23f5-0920-42f2-b3cc-a672191fe7ff admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:01.282366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c6c07-3ebf-41ee-aaa6-b1ad060060ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:01.283123 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c6c07-3ebf-41ee-aaa6-b1ad060060ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:01.283626 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c6c07-3ebf-41ee-aaa6-b1ad060060ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:01.283917 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c6c07-3ebf-41ee-aaa6-b1ad060060ab admin admin] PATH_INFO: `/v3/users/91ca8b5941a84fa580672dea3d78a9b9` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:01.288744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c6c07-3ebf-41ee-aaa6-b1ad060060ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ca8b5941a84fa580672dea3d78a9b9)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:01.290801 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c6c07-3ebf-41ee-aaa6-b1ad060060ab admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:01.409033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-49f806c0-f859-4969-9272-48bcaff5400f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aba5dc96d6994997ab1df353053e15d9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:01.465810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-49f806c0-f859-4969-9272-48bcaff5400f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aba5dc96d6994997ab1df353053e15d9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:01.483204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-49f806c0-f859-4969-9272-48bcaff5400f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aba5dc96d6994997ab1df353053e15d9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:01.547123 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d95c6c07-3ebf-41ee-aaa6-b1ad060060ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ca8b5941a84fa580672dea3d78a9b9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:01.595760 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-49f806c0-f859-4969-9272-48bcaff5400f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aba5dc96d6994997ab1df353053e15d9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:01.595760 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-49f806c0-f859-4969-9272-48bcaff5400f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aba5dc96d6994997ab1df353053e15d9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:01.600848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 836/1660] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:06:00 2022] DELETE /identity/v3/users/aba5dc96d6994997ab1df353053e15d9 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:01.602353 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c237fc-8132-4e11-bc9e-aa5d6fb32f07 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:01.609834 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c237fc-8132-4e11-bc9e-aa5d6fb32f07 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:01.610300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c237fc-8132-4e11-bc9e-aa5d6fb32f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:01.632183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d95c6c07-3ebf-41ee-aaa6-b1ad060060ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ca8b5941a84fa580672dea3d78a9b9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:01.660812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d95c6c07-3ebf-41ee-aaa6-b1ad060060ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ca8b5941a84fa580672dea3d78a9b9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:01.734025 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d95c6c07-3ebf-41ee-aaa6-b1ad060060ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ca8b5941a84fa580672dea3d78a9b9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:01.734819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d95c6c07-3ebf-41ee-aaa6-b1ad060060ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ca8b5941a84fa580672dea3d78a9b9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:01.739990 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-17c237fc-8132-4e11-bc9e-aa5d6fb32f07 None None] MFA Rules not processed for user `f04baf2739c0430e8514faa2c42de104`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:01.745771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-17c237fc-8132-4e11-bc9e-aa5d6fb32f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:01.745999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 825/1661] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:06:01 2022] DELETE /identity/v3/users/91ca8b5941a84fa580672dea3d78a9b9 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:01.748016 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d62b2-2384-4f15-aacd-75c25ce7f1ba None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:01.748391 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d62b2-2384-4f15-aacd-75c25ce7f1ba None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:01.748775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d62b2-2384-4f15-aacd-75c25ce7f1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:01.821811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:01.849940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-188d62b2-2384-4f15-aacd-75c25ce7f1ba None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:01.849940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-188d62b2-2384-4f15-aacd-75c25ce7f1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:01.952570 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 837/1662] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:01.953851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 826/1663] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:01.954382 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c237fc-8132-4e11-bc9e-aa5d6fb32f07 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:01.955437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-17c237fc-8132-4e11-bc9e-aa5d6fb32f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:01.996454 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c237fc-8132-4e11-bc9e-aa5d6fb32f07 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:02.009997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4b8a4-90a9-46a2-bce4-52208fba9ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:02.010634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4b8a4-90a9-46a2-bce4-52208fba9ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:02.010913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4b8a4-90a9-46a2-bce4-52208fba9ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:02.011186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4b8a4-90a9-46a2-bce4-52208fba9ed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:02.011938 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4b8a4-90a9-46a2-bce4-52208fba9ed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:02.015866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4b8a4-90a9-46a2-bce4-52208fba9ed7 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:02.039010 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d62b2-2384-4f15-aacd-75c25ce7f1ba None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:02.040553 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-188d62b2-2384-4f15-aacd-75c25ce7f1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:02.155387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 838/1664] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:06:01 2022] POST /identity/v3/projects => generated 365 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:06:02.159765 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4b8a4-90a9-46a2-bce4-52208fba9ed7 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:02.248516 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d62b2-2384-4f15-aacd-75c25ce7f1ba None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:02.285195 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:02.355286 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4b8a4-90a9-46a2-bce4-52208fba9ed7 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:02.368084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4b18c-ac23-4a8b-9825-cc2bb23ff520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:02.368586 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4b18c-ac23-4a8b-9825-cc2bb23ff520 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:02.369026 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4b18c-ac23-4a8b-9825-cc2bb23ff520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:02.369824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4b18c-ac23-4a8b-9825-cc2bb23ff520 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:02.370056 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4b18c-ac23-4a8b-9825-cc2bb23ff520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:02.372538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4b18c-ac23-4a8b-9825-cc2bb23ff520 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:02.392114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838434dc-bf9f-4b6b-939c-9a7045f2f398 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:02.392795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838434dc-bf9f-4b6b-939c-9a7045f2f398 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:02.394161 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838434dc-bf9f-4b6b-939c-9a7045f2f398 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:02.398614 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838434dc-bf9f-4b6b-939c-9a7045f2f398 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:02.419605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838434dc-bf9f-4b6b-939c-9a7045f2f398 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:02.421559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838434dc-bf9f-4b6b-939c-9a7045f2f398 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:02.448652 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 827/1665] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:02.539090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 839/1666] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:06:02 2022] POST /identity/v3/users => generated 289 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:06:02.542902 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838434dc-bf9f-4b6b-939c-9a7045f2f398 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:02.559083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838434dc-bf9f-4b6b-939c-9a7045f2f398 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:02.577573 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71017c01-215f-42ed-94c5-18409817276e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:02.578941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71017c01-215f-42ed-94c5-18409817276e admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:02.579410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71017c01-215f-42ed-94c5-18409817276e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:02.580082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71017c01-215f-42ed-94c5-18409817276e admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:02.580465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71017c01-215f-42ed-94c5-18409817276e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:02.588109 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71017c01-215f-42ed-94c5-18409817276e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:02.631591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 828/1667] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:06:02.632886 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4b18c-ac23-4a8b-9825-cc2bb23ff520 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:02.645757 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4b18c-ac23-4a8b-9825-cc2bb23ff520 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:02.658610 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c34ab-9dae-45a2-8c8c-e10dc332c15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:02.660650 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c34ab-9dae-45a2-8c8c-e10dc332c15a admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:02.661139 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c34ab-9dae-45a2-8c8c-e10dc332c15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:02.662398 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c34ab-9dae-45a2-8c8c-e10dc332c15a admin admin] PATH_INFO: `/v3/projects/696f03908243454d88e1beb80a580e83/users/8cd9e39ac2e3488f8fabb78e8807399e/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:02.705652 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c34ab-9dae-45a2-8c8c-e10dc332c15a admin admin] RBAC: Authorizing `identity:create_grant(project_id=696f03908243454d88e1beb80a580e83, user_id=8cd9e39ac2e3488f8fabb78e8807399e, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:02.709669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c34ab-9dae-45a2-8c8c-e10dc332c15a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:02.812261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 840/1668] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:06:02 2022] PUT /identity/v3/projects/696f03908243454d88e1beb80a580e83/users/8cd9e39ac2e3488f8fabb78e8807399e/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:02.816484 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71017c01-215f-42ed-94c5-18409817276e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:02.847071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71017c01-215f-42ed-94c5-18409817276e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:03.027760 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9923971-5989-4c7d-b45c-670e5d5cc5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:03.028400 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9923971-5989-4c7d-b45c-670e5d5cc5df admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:03.029973 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9923971-5989-4c7d-b45c-670e5d5cc5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:03.029973 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9923971-5989-4c7d-b45c-670e5d5cc5df admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:03.029973 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9923971-5989-4c7d-b45c-670e5d5cc5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:03.033093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9923971-5989-4c7d-b45c-670e5d5cc5df admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:03.057024 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 829/1669] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:06:03.061138 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c34ab-9dae-45a2-8c8c-e10dc332c15a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:03.080265 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c34ab-9dae-45a2-8c8c-e10dc332c15a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:03.096007 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8feb35-7bb1-4da3-9e40-16b70c774728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:03.096293 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8feb35-7bb1-4da3-9e40-16b70c774728 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:03.096701 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8feb35-7bb1-4da3-9e40-16b70c774728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:03.097096 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8feb35-7bb1-4da3-9e40-16b70c774728 admin admin] PATH_INFO: `/v3/projects/696f03908243454d88e1beb80a580e83/users/8cd9e39ac2e3488f8fabb78e8807399e/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:03.101468 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8feb35-7bb1-4da3-9e40-16b70c774728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=696f03908243454d88e1beb80a580e83, user_id=8cd9e39ac2e3488f8fabb78e8807399e, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:03.105001 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8feb35-7bb1-4da3-9e40-16b70c774728 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:03.137523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 841/1670] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:06:03 2022] PUT /identity/v3/projects/696f03908243454d88e1beb80a580e83/users/8cd9e39ac2e3488f8fabb78e8807399e/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:03.150232 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc028ab-a900-47f3-94a7-4c535ffbeb8a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:03.150232 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc028ab-a900-47f3-94a7-4c535ffbeb8a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:03.150232 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc028ab-a900-47f3-94a7-4c535ffbeb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:03.169172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe6816-3ac2-4921-bb4c-cfcb4fe816b6 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:03.169172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe6816-3ac2-4921-bb4c-cfcb4fe816b6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:03.169172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe6816-3ac2-4921-bb4c-cfcb4fe816b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:03.228646 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-cfbe6816-3ac2-4921-bb4c-cfcb4fe816b6 None None] MFA Rules not processed for user `d9d8955f8bdb499eab93afb503505ccd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:03.230186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cfbe6816-3ac2-4921-bb4c-cfcb4fe816b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:03.241535 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-5cc028ab-a900-47f3-94a7-4c535ffbeb8a None None] MFA Rules not processed for user `8cd9e39ac2e3488f8fabb78e8807399e`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:03.246425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5cc028ab-a900-47f3-94a7-4c535ffbeb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:03.360093 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:03.425899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 842/1671] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:03 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:03.457905 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 830/1672] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:03.465703 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe6816-3ac2-4921-bb4c-cfcb4fe816b6 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:03.467246 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cfbe6816-3ac2-4921-bb4c-cfcb4fe816b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:03.623164 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe6816-3ac2-4921-bb4c-cfcb4fe816b6 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:03.639864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6900c-6542-4f60-95ca-508afe890801 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:03.640620 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6900c-6542-4f60-95ca-508afe890801 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:03.641046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6900c-6542-4f60-95ca-508afe890801 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:03.641475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6900c-6542-4f60-95ca-508afe890801 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:03.660434 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6900c-6542-4f60-95ca-508afe890801 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:03.662272 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6900c-6542-4f60-95ca-508afe890801 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:03.688996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 843/1673] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:04.004330 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc028ab-a900-47f3-94a7-4c535ffbeb8a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:04.007280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5cc028ab-a900-47f3-94a7-4c535ffbeb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:04.075910 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6900c-6542-4f60-95ca-508afe890801 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:04.086492 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6900c-6542-4f60-95ca-508afe890801 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:04.098627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9e83b-c12d-4eb5-ac21-1398894dc55c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:04.099759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9e83b-c12d-4eb5-ac21-1398894dc55c service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:04.103964 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:04.104917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9e83b-c12d-4eb5-ac21-1398894dc55c service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:04.105250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9e83b-c12d-4eb5-ac21-1398894dc55c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:04.213482 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc028ab-a900-47f3-94a7-4c535ffbeb8a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:04.221110 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9fe652-7a48-4d52-a6ab-f0d69e45a8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:04.222398 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fe652-7a48-4d52-a6ab-f0d69e45a8a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:04.222764 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fe652-7a48-4d52-a6ab-f0d69e45a8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:04.223508 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fe652-7a48-4d52-a6ab-f0d69e45a8a9 admin admin] PATH_INFO: `/v3/projects/efcd7551351e41f7895b38ca6ad7271b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:04.233622 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9fe652-7a48-4d52-a6ab-f0d69e45a8a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efcd7551351e41f7895b38ca6ad7271b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:04.236016 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9fe652-7a48-4d52-a6ab-f0d69e45a8a9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:04.293087 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d9e83b-c12d-4eb5-ac21-1398894dc55c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:04.294828 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d9e83b-c12d-4eb5-ac21-1398894dc55c service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:04.320243 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 844/1674] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:04.441026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3c9fe652-7a48-4d52-a6ab-f0d69e45a8a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efcd7551351e41f7895b38ca6ad7271b', 'request_id': 'req-3c9fe652-7a48-4d52-a6ab-f0d69e45a8a9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:04.517104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9e83b-c12d-4eb5-ac21-1398894dc55c service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:04.518790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-09d9e83b-c12d-4eb5-ac21-1398894dc55c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:04.547514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9e83b-c12d-4eb5-ac21-1398894dc55c service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:04.584942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3c9fe652-7a48-4d52-a6ab-f0d69e45a8a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efcd7551351e41f7895b38ca6ad7271b', 'request_id': 'req-3c9fe652-7a48-4d52-a6ab-f0d69e45a8a9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:04.592162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3c9fe652-7a48-4d52-a6ab-f0d69e45a8a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efcd7551351e41f7895b38ca6ad7271b', 'request_id': 'req-3c9fe652-7a48-4d52-a6ab-f0d69e45a8a9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:04.620416 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3c9fe652-7a48-4d52-a6ab-f0d69e45a8a9 admin admin] The token cache is being invalidate because project efcd7551351e41f7895b38ca6ad7271b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:06:04.621296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3c9fe652-7a48-4d52-a6ab-f0d69e45a8a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:04.622361 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3c9fe652-7a48-4d52-a6ab-f0d69e45a8a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:04.635244 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 831/1675] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:06:04 2022] DELETE /identity/v3/projects/efcd7551351e41f7895b38ca6ad7271b => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:04.637889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caac5d8-85ac-4798-b90b-f323e22d79e3 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:04.637889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caac5d8-85ac-4798-b90b-f323e22d79e3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:04.637889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caac5d8-85ac-4798-b90b-f323e22d79e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:04.694296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-7caac5d8-85ac-4798-b90b-f323e22d79e3 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:04.697881 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7caac5d8-85ac-4798-b90b-f323e22d79e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:04.702246 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857ff59-de48-4739-b8dd-eb86c667dc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:04.702917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857ff59-de48-4739-b8dd-eb86c667dc43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:04.704877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857ff59-de48-4739-b8dd-eb86c667dc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:04.705659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857ff59-de48-4739-b8dd-eb86c667dc43 admin admin] PATH_INFO: `/v3/projects/14ea26a132e844ec9d7c02540909e8cf` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:04.712535 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857ff59-de48-4739-b8dd-eb86c667dc43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14ea26a132e844ec9d7c02540909e8cf)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:04.718548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857ff59-de48-4739-b8dd-eb86c667dc43 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:04.719900 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 832/1676] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:04.728460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56678e5e-1e39-4919-93e7-571c2af394dd None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:04.732061 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56678e5e-1e39-4919-93e7-571c2af394dd None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:04.732783 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56678e5e-1e39-4919-93e7-571c2af394dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:04.801835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-56678e5e-1e39-4919-93e7-571c2af394dd None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:04.803554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-56678e5e-1e39-4919-93e7-571c2af394dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:04.825718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 833/1677] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:04.833109 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56678e5e-1e39-4919-93e7-571c2af394dd None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:04.869887 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56678e5e-1e39-4919-93e7-571c2af394dd None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:04.875009 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:04.894608 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e7cda-b064-476d-84a0-1374a2a711e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:04.895393 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e7cda-b064-476d-84a0-1374a2a711e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:04.895796 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e7cda-b064-476d-84a0-1374a2a711e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:04.896232 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e7cda-b064-476d-84a0-1374a2a711e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:04.896991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e7cda-b064-476d-84a0-1374a2a711e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:06:04.897411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e7cda-b064-476d-84a0-1374a2a711e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:04.899225 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e7cda-b064-476d-84a0-1374a2a711e3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:04.900509 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1857ff59-de48-4739-b8dd-eb86c667dc43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14ea26a132e844ec9d7c02540909e8cf', 'request_id': 'req-1857ff59-de48-4739-b8dd-eb86c667dc43'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:04.955094 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 834/1678] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 12:06:04.956886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e7cda-b064-476d-84a0-1374a2a711e3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:04.979603 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1857ff59-de48-4739-b8dd-eb86c667dc43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14ea26a132e844ec9d7c02540909e8cf', 'request_id': 'req-1857ff59-de48-4739-b8dd-eb86c667dc43'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:04.989611 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1857ff59-de48-4739-b8dd-eb86c667dc43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14ea26a132e844ec9d7c02540909e8cf', 'request_id': 'req-1857ff59-de48-4739-b8dd-eb86c667dc43'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:05.004140 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e7cda-b064-476d-84a0-1374a2a711e3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:05.021241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1857ff59-de48-4739-b8dd-eb86c667dc43 admin admin] The token cache is being invalidate because project 14ea26a132e844ec9d7c02540909e8cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:06:05.021584 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1857ff59-de48-4739-b8dd-eb86c667dc43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:05.024332 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1857ff59-de48-4739-b8dd-eb86c667dc43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:05.029033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 845/1679] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:06:04 2022] DELETE /identity/v3/projects/14ea26a132e844ec9d7c02540909e8cf => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:06:05.040652 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9d68d-c298-4f6d-a782-2f5da36851c7 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:05.040857 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9d68d-c298-4f6d-a782-2f5da36851c7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:05.041192 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9d68d-c298-4f6d-a782-2f5da36851c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:05.084318 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0ac9d68d-c298-4f6d-a782-2f5da36851c7 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:05.086230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0ac9d68d-c298-4f6d-a782-2f5da36851c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:05.221428 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75e1bc-8218-4b27-a4f9-ae49fb97cbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:05.222381 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75e1bc-8218-4b27-a4f9-ae49fb97cbef admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:05.222779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75e1bc-8218-4b27-a4f9-ae49fb97cbef admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:05.223189 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75e1bc-8218-4b27-a4f9-ae49fb97cbef admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:05.223983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75e1bc-8218-4b27-a4f9-ae49fb97cbef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:05.226469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75e1bc-8218-4b27-a4f9-ae49fb97cbef admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:05.376901 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 846/1680] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:05.380308 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 835/1681] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:06:04 2022] POST /identity/v3/projects => generated 377 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:06:05.384414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9d68d-c298-4f6d-a782-2f5da36851c7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:05.386018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0ac9d68d-c298-4f6d-a782-2f5da36851c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:05.393776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9da28-f1a3-41aa-ac70-8dbfa3c9be7d None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:05.394745 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9da28-f1a3-41aa-ac70-8dbfa3c9be7d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:05.395326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9da28-f1a3-41aa-ac70-8dbfa3c9be7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:05.410130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9d68d-c298-4f6d-a782-2f5da36851c7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:05.433356 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-30f9da28-f1a3-41aa-ac70-8dbfa3c9be7d None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:05.434871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-30f9da28-f1a3-41aa-ac70-8dbfa3c9be7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:05.469932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:05.548563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0526ac-e2a9-4646-8fc9-1e31d140d7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:05.549227 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0526ac-e2a9-4646-8fc9-1e31d140d7f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:05.550093 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0526ac-e2a9-4646-8fc9-1e31d140d7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:05.550648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0526ac-e2a9-4646-8fc9-1e31d140d7f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:05.551267 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0526ac-e2a9-4646-8fc9-1e31d140d7f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:05.555036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0526ac-e2a9-4646-8fc9-1e31d140d7f1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:05.597952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 836/1682] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:05.602175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9da28-f1a3-41aa-ac70-8dbfa3c9be7d None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:05.621772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9da28-f1a3-41aa-ac70-8dbfa3c9be7d None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:05.635432 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b894c-c5c5-475a-9fed-08825eff79a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:05.636422 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b894c-c5c5-475a-9fed-08825eff79a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:05.636790 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b894c-c5c5-475a-9fed-08825eff79a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:05.637176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b894c-c5c5-475a-9fed-08825eff79a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:05.638191 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b894c-c5c5-475a-9fed-08825eff79a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:06:05.638821 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b894c-c5c5-475a-9fed-08825eff79a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:05.644814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b894c-c5c5-475a-9fed-08825eff79a0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:05.661113 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 837/1683] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 12:06:05.666266 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b894c-c5c5-475a-9fed-08825eff79a0 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:05.676530 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 847/1684] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:06:05 2022] POST /identity/v3/users => generated 289 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:06:05.685244 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b894c-c5c5-475a-9fed-08825eff79a0 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:05.685679 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0526ac-e2a9-4646-8fc9-1e31d140d7f1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:05.695594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946c92e-b3d4-4e2b-9b82-fbf4fe41dc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:05.696596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946c92e-b3d4-4e2b-9b82-fbf4fe41dc20 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:05.697002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946c92e-b3d4-4e2b-9b82-fbf4fe41dc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:05.698150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946c92e-b3d4-4e2b-9b82-fbf4fe41dc20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:05.699123 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0526ac-e2a9-4646-8fc9-1e31d140d7f1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:05.699291 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946c92e-b3d4-4e2b-9b82-fbf4fe41dc20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:05.702043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946c92e-b3d4-4e2b-9b82-fbf4fe41dc20 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:05.717344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c383207-b894-4946-a9e7-b91b709b6f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:05.717344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c383207-b894-4946-a9e7-b91b709b6f9f admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:05.717344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c383207-b894-4946-a9e7-b91b709b6f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:05.717344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c383207-b894-4946-a9e7-b91b709b6f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:05.718132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c383207-b894-4946-a9e7-b91b709b6f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:05.733054 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c383207-b894-4946-a9e7-b91b709b6f9f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:05.750496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 848/1685] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:06:05.754255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c383207-b894-4946-a9e7-b91b709b6f9f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:05.773908 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c383207-b894-4946-a9e7-b91b709b6f9f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:05.786817 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03080a58-f781-4639-99b8-eca15dfa8ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:05.787462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03080a58-f781-4639-99b8-eca15dfa8ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:05.787750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03080a58-f781-4639-99b8-eca15dfa8ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:05.788040 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03080a58-f781-4639-99b8-eca15dfa8ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:05.788672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03080a58-f781-4639-99b8-eca15dfa8ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:05.790413 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03080a58-f781-4639-99b8-eca15dfa8ae6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:05.809475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 838/1686] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:06:05 2022] POST /identity/v3/projects => generated 355 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:06:05.815486 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946c92e-b3d4-4e2b-9b82-fbf4fe41dc20 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:05.819151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 849/1687] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:06:05.820430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03080a58-f781-4639-99b8-eca15dfa8ae6 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:05.838136 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03080a58-f781-4639-99b8-eca15dfa8ae6 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:05.903754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:05.994794 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7129d-faf6-41bb-9a2f-38d35f5b3359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:05.995501 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7129d-faf6-41bb-9a2f-38d35f5b3359 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:05.995790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7129d-faf6-41bb-9a2f-38d35f5b3359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:05.996072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7129d-faf6-41bb-9a2f-38d35f5b3359 admin admin] PATH_INFO: `/v3/projects/c49e18812eae47d4977865566fb64f1d/users/4ed5c2a1c6a84caf92b213b9efd481e4/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:06.023217 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946c92e-b3d4-4e2b-9b82-fbf4fe41dc20 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:06.034770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41423522-204c-49b8-8749-903275a181ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:06.034770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41423522-204c-49b8-8749-903275a181ef admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:06.035634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41423522-204c-49b8-8749-903275a181ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:06.036101 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41423522-204c-49b8-8749-903275a181ef admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:06.036953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41423522-204c-49b8-8749-903275a181ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:06.038461 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7129d-faf6-41bb-9a2f-38d35f5b3359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c49e18812eae47d4977865566fb64f1d, user_id=4ed5c2a1c6a84caf92b213b9efd481e4, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:06.039360 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41423522-204c-49b8-8749-903275a181ef admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:06.041253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7129d-faf6-41bb-9a2f-38d35f5b3359 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:06.133719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 850/1688] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:06:05 2022] PUT /identity/v3/projects/c49e18812eae47d4977865566fb64f1d/users/4ed5c2a1c6a84caf92b213b9efd481e4/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:06.146899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7d33f-d0c6-424a-bd0d-9c7402e184be None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:06.147195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7d33f-d0c6-424a-bd0d-9c7402e184be None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:06.147670 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7d33f-d0c6-424a-bd0d-9c7402e184be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:06.158656 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 839/1689] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:06:05 2022] POST /identity/v3/users => generated 278 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:06:06.165150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41423522-204c-49b8-8749-903275a181ef admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:06.197720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41423522-204c-49b8-8749-903275a181ef admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:06.261422 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-37f7d33f-d0c6-424a-bd0d-9c7402e184be None None] MFA Rules not processed for user `4ed5c2a1c6a84caf92b213b9efd481e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:06.262823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-37f7d33f-d0c6-424a-bd0d-9c7402e184be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:06.401350 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a222f7-9213-4183-8491-e5f781feaadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:06.402059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a222f7-9213-4183-8491-e5f781feaadb admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:06.402368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a222f7-9213-4183-8491-e5f781feaadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:06.402920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a222f7-9213-4183-8491-e5f781feaadb admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:06.403711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a222f7-9213-4183-8491-e5f781feaadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:06.407323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a222f7-9213-4183-8491-e5f781feaadb admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:06.431922 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 840/1690] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:06:06.434220 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a222f7-9213-4183-8491-e5f781feaadb admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:06.462981 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a222f7-9213-4183-8491-e5f781feaadb admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:06.477276 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 851/1691] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:06.481301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce5148-c7cb-44cc-b918-6b24f235e5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:06.485681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce5148-c7cb-44cc-b918-6b24f235e5a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:06.486243 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce5148-c7cb-44cc-b918-6b24f235e5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:06.486748 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce5148-c7cb-44cc-b918-6b24f235e5a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:06.487791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce5148-c7cb-44cc-b918-6b24f235e5a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:06.493633 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7d33f-d0c6-424a-bd0d-9c7402e184be None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:06.493734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce5148-c7cb-44cc-b918-6b24f235e5a0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:06.517187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:06.530457 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 841/1692] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:06:06.531619 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7d33f-d0c6-424a-bd0d-9c7402e184be None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:06.532320 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce5148-c7cb-44cc-b918-6b24f235e5a0 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:06.541479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703a393-fa0a-4735-83cf-beb8882ad809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:06.542527 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703a393-fa0a-4735-83cf-beb8882ad809 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:06.543032 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703a393-fa0a-4735-83cf-beb8882ad809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:06.543556 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703a393-fa0a-4735-83cf-beb8882ad809 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:06.544478 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703a393-fa0a-4735-83cf-beb8882ad809 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:06.547559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703a393-fa0a-4735-83cf-beb8882ad809 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:06.551661 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce5148-c7cb-44cc-b918-6b24f235e5a0 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:06.560118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f77c9-9ab6-4957-8cbf-36354413b63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:06.562772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f77c9-9ab6-4957-8cbf-36354413b63e admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:06.563196 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f77c9-9ab6-4957-8cbf-36354413b63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:06.568209 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f77c9-9ab6-4957-8cbf-36354413b63e admin admin] PATH_INFO: `/v3/projects/0c69b707fee641568ba3e1de7d5c4de1/users/a6f788f0db6043d1a7be3068408a68a6/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:06.597635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f77c9-9ab6-4957-8cbf-36354413b63e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c69b707fee641568ba3e1de7d5c4de1, user_id=a6f788f0db6043d1a7be3068408a68a6, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:06.615428 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f77c9-9ab6-4957-8cbf-36354413b63e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:06.688742 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 852/1693] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:06:06 2022] POST /identity/v3/projects => generated 373 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:06:06.690019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703a393-fa0a-4735-83cf-beb8882ad809 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:06.751386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 842/1694] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:06:06 2022] PUT /identity/v3/projects/0c69b707fee641568ba3e1de7d5c4de1/users/a6f788f0db6043d1a7be3068408a68a6/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:06.763924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f927d4a-6104-4d3f-aa0a-bf8e23d1418d None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:06.764355 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f927d4a-6104-4d3f-aa0a-bf8e23d1418d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:06.764715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f927d4a-6104-4d3f-aa0a-bf8e23d1418d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:06.838772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-5f927d4a-6104-4d3f-aa0a-bf8e23d1418d None None] MFA Rules not processed for user `a6f788f0db6043d1a7be3068408a68a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:06.840113 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5f927d4a-6104-4d3f-aa0a-bf8e23d1418d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:06.842506 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703a393-fa0a-4735-83cf-beb8882ad809 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:06.892410 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:07.000499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 843/1695] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:06:07.007284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f927d4a-6104-4d3f-aa0a-bf8e23d1418d None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:07.011364 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383811e-2c3b-49bb-b82f-cfeb00de6411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:07.012149 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383811e-2c3b-49bb-b82f-cfeb00de6411 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:07.012868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383811e-2c3b-49bb-b82f-cfeb00de6411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:07.013337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383811e-2c3b-49bb-b82f-cfeb00de6411 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:07.014498 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383811e-2c3b-49bb-b82f-cfeb00de6411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:07.020802 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383811e-2c3b-49bb-b82f-cfeb00de6411 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:07.031087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f927d4a-6104-4d3f-aa0a-bf8e23d1418d None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:07.045107 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a948ab1-5736-4c3e-b4aa-962fb49b2938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:07.045819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a948ab1-5736-4c3e-b4aa-962fb49b2938 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:07.046146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a948ab1-5736-4c3e-b4aa-962fb49b2938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:07.046476 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a948ab1-5736-4c3e-b4aa-962fb49b2938 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:07.047236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a948ab1-5736-4c3e-b4aa-962fb49b2938 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:07.050156 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a948ab1-5736-4c3e-b4aa-962fb49b2938 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:07.154717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 853/1696] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:06:06 2022] POST /identity/v3/users => generated 293 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:06:07.158196 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383811e-2c3b-49bb-b82f-cfeb00de6411 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:07.200292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 844/1697] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:06:07 2022] POST /identity/v3/projects => generated 355 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:06:07.202739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383811e-2c3b-49bb-b82f-cfeb00de6411 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:07.203379 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a948ab1-5736-4c3e-b4aa-962fb49b2938 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:07.370594 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ae6ad-7703-439e-8aca-fea058cb33b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:07.370594 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ae6ad-7703-439e-8aca-fea058cb33b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:07.370594 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ae6ad-7703-439e-8aca-fea058cb33b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:07.377174 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ae6ad-7703-439e-8aca-fea058cb33b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:07.379052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ae6ad-7703-439e-8aca-fea058cb33b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:07.381465 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ae6ad-7703-439e-8aca-fea058cb33b6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:07.385036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a948ab1-5736-4c3e-b4aa-962fb49b2938 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:07.412087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa64c8-9cd6-45c6-a228-566a7a588ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:07.412964 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa64c8-9cd6-45c6-a228-566a7a588ccd admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:07.413145 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa64c8-9cd6-45c6-a228-566a7a588ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:07.413393 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa64c8-9cd6-45c6-a228-566a7a588ccd admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:07.421785 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfa64c8-9cd6-45c6-a228-566a7a588ccd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:07.429406 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfa64c8-9cd6-45c6-a228-566a7a588ccd admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:07.446289 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 854/1698] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:06:07.452727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468f085-3a29-4f1b-bd45-4366849dc860 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:07.462465 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468f085-3a29-4f1b-bd45-4366849dc860 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:07.462465 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468f085-3a29-4f1b-bd45-4366849dc860 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:07.462465 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 855/1699] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 14 12:06:07 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:06:07.462465 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468f085-3a29-4f1b-bd45-4366849dc860 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:07.497668 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468f085-3a29-4f1b-bd45-4366849dc860 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:07.510059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bb5cc-6ace-4013-bdda-f90162205da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:07.510750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bb5cc-6ace-4013-bdda-f90162205da4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:07.511033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bb5cc-6ace-4013-bdda-f90162205da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:07.511312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bb5cc-6ace-4013-bdda-f90162205da4 admin admin] PATH_INFO: `/v3/projects/5770f7450c71491e9b94ce392e5489ea/users/91a9ec8d71824332838b3360bc90f800/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:07.554999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 845/1700] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:06:07 2022] POST /identity/v3/users => generated 284 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:06:07.559801 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa64c8-9cd6-45c6-a228-566a7a588ccd admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:07.610924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa64c8-9cd6-45c6-a228-566a7a588ccd admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:07.618531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bb5cc-6ace-4013-bdda-f90162205da4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5770f7450c71491e9b94ce392e5489ea, user_id=91a9ec8d71824332838b3360bc90f800, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:07.633281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bb5cc-6ace-4013-bdda-f90162205da4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:07.639601 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed62f6-5229-40d5-9e7a-085b7ca5b780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:07.641249 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed62f6-5229-40d5-9e7a-085b7ca5b780 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:07.641657 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed62f6-5229-40d5-9e7a-085b7ca5b780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:07.641973 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed62f6-5229-40d5-9e7a-085b7ca5b780 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:07.642983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed62f6-5229-40d5-9e7a-085b7ca5b780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:07.654918 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed62f6-5229-40d5-9e7a-085b7ca5b780 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:07.708755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 846/1701] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:06:07.712252 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e545e-1fff-4f34-99d9-762bbd4a3773 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:07.712555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e545e-1fff-4f34-99d9-762bbd4a3773 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:07.712838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e545e-1fff-4f34-99d9-762bbd4a3773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:07.737024 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 856/1702] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:06:07 2022] PUT /identity/v3/projects/5770f7450c71491e9b94ce392e5489ea/users/91a9ec8d71824332838b3360bc90f800/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:07.744930 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bb5cc-6ace-4013-bdda-f90162205da4 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:07.803012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8d7e545e-1fff-4f34-99d9-762bbd4a3773 None None] MFA Rules not processed for user `0ca10dd7ed714edb8b87017deecc41b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:07.804174 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8d7e545e-1fff-4f34-99d9-762bbd4a3773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:07.867549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:07.957056 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bb5cc-6ace-4013-bdda-f90162205da4 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:07.966392 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c93af-02dd-4eb3-a4ad-b4948be760ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:07.967090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c93af-02dd-4eb3-a4ad-b4948be760ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:07.967377 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c93af-02dd-4eb3-a4ad-b4948be760ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:07.967661 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c93af-02dd-4eb3-a4ad-b4948be760ba admin admin] PATH_INFO: `/v3/projects/67bc496988864f4286330e1808e42a5f/users/dff34d763447487b9dfb46c0707b738b/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:07.988981 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c93af-02dd-4eb3-a4ad-b4948be760ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=67bc496988864f4286330e1808e42a5f, user_id=dff34d763447487b9dfb46c0707b738b, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:07.992369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c93af-02dd-4eb3-a4ad-b4948be760ba admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:08.008646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 847/1703] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 14 12:06:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:08.009675 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e545e-1fff-4f34-99d9-762bbd4a3773 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:08.027988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e545e-1fff-4f34-99d9-762bbd4a3773 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:08.039609 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b8d44-b7f5-46a5-9af7-2922da0e2e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:08.040718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b8d44-b7f5-46a5-9af7-2922da0e2e61 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:08.040918 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b8d44-b7f5-46a5-9af7-2922da0e2e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:08.041087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b8d44-b7f5-46a5-9af7-2922da0e2e61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:08.041736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b8d44-b7f5-46a5-9af7-2922da0e2e61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:08.048665 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b8d44-b7f5-46a5-9af7-2922da0e2e61 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:08.079513 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 848/1704] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:06:08.080545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b8d44-b7f5-46a5-9af7-2922da0e2e61 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:08.083472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0f1b8d44-b7f5-46a5-9af7-2922da0e2e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:08.108778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b8d44-b7f5-46a5-9af7-2922da0e2e61 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:08.119137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e3d8b-16d3-43f7-8b02-f092f564304d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca10dd7ed714edb8b87017deecc41b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:08.119778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e3d8b-16d3-43f7-8b02-f092f564304d service placement] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:08.120060 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e3d8b-16d3-43f7-8b02-f092f564304d service placement] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:08.120335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e3d8b-16d3-43f7-8b02-f092f564304d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:08.135378 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 857/1705] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:06:07 2022] PUT /identity/v3/projects/67bc496988864f4286330e1808e42a5f/users/dff34d763447487b9dfb46c0707b738b/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:08.141257 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c93af-02dd-4eb3-a4ad-b4948be760ba admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:08.141530 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e3d8b-16d3-43f7-8b02-f092f564304d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:08.142097 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e3d8b-16d3-43f7-8b02-f092f564304d service placement] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:08.158642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 849/1706] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:08.170365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e3d8b-16d3-43f7-8b02-f092f564304d service placement] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:08.194078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e3d8b-16d3-43f7-8b02-f092f564304d service placement] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:08.206893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:08.288586 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c93af-02dd-4eb3-a4ad-b4948be760ba admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:08.296763 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9d859-4333-4fc1-87be-6342aa2c687f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:08.297524 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9d859-4333-4fc1-87be-6342aa2c687f admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:08.297811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9d859-4333-4fc1-87be-6342aa2c687f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:08.298120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9d859-4333-4fc1-87be-6342aa2c687f admin admin] PATH_INFO: `/v3/projects/5770f7450c71491e9b94ce392e5489ea/users/91a9ec8d71824332838b3360bc90f800/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:08.304455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9d859-4333-4fc1-87be-6342aa2c687f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5770f7450c71491e9b94ce392e5489ea, user_id=91a9ec8d71824332838b3360bc90f800, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:08.307962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9d859-4333-4fc1-87be-6342aa2c687f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:08.333147 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a157e81-7794-423a-b806-9e7000238bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:08.338287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a157e81-7794-423a-b806-9e7000238bba admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:08.339024 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a157e81-7794-423a-b806-9e7000238bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:08.340662 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a157e81-7794-423a-b806-9e7000238bba admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:08.342000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a157e81-7794-423a-b806-9e7000238bba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:08.344128 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a157e81-7794-423a-b806-9e7000238bba admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:08.350308 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 858/1707] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:06:08 2022] PUT /identity/v3/projects/5770f7450c71491e9b94ce392e5489ea/users/91a9ec8d71824332838b3360bc90f800/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:08.352017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc653080-6a4f-4aff-afe0-b5033fb0143d None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:08.352314 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc653080-6a4f-4aff-afe0-b5033fb0143d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:08.352596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc653080-6a4f-4aff-afe0-b5033fb0143d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:08.370923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 850/1708] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:06:08.372409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a157e81-7794-423a-b806-9e7000238bba admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:08.390176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a157e81-7794-423a-b806-9e7000238bba admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:08.415388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-cc653080-6a4f-4aff-afe0-b5033fb0143d None None] MFA Rules not processed for user `91a9ec8d71824332838b3360bc90f800`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:08.422101 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cc653080-6a4f-4aff-afe0-b5033fb0143d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:08.521276 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c71df9-2f2e-43f0-b8c3-f32dbaaea1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:08.522002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c71df9-2f2e-43f0-b8c3-f32dbaaea1be admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:08.522295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c71df9-2f2e-43f0-b8c3-f32dbaaea1be admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:08.522574 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c71df9-2f2e-43f0-b8c3-f32dbaaea1be admin admin] PATH_INFO: `/v3/projects/67bc496988864f4286330e1808e42a5f/users/dff34d763447487b9dfb46c0707b738b/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:08.526525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c71df9-2f2e-43f0-b8c3-f32dbaaea1be admin admin] RBAC: Authorizing `identity:create_grant(project_id=67bc496988864f4286330e1808e42a5f, user_id=dff34d763447487b9dfb46c0707b738b, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:08.529271 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c71df9-2f2e-43f0-b8c3-f32dbaaea1be admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:08.567941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 851/1709] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:06:08 2022] PUT /identity/v3/projects/67bc496988864f4286330e1808e42a5f/users/dff34d763447487b9dfb46c0707b738b/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:08.567941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8ed68-5c3f-470d-9856-b6b1c21dffbc None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:08.567941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8ed68-5c3f-470d-9856-b6b1c21dffbc None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:08.567941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8ed68-5c3f-470d-9856-b6b1c21dffbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:08.595376 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 859/1710] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:08.610385 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65737b5e-c9ce-49a5-9908-4c81174e764e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:08.610678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65737b5e-c9ce-49a5-9908-4c81174e764e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:08.610959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65737b5e-c9ce-49a5-9908-4c81174e764e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:08.648044 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-64c8ed68-5c3f-470d-9856-b6b1c21dffbc None None] MFA Rules not processed for user `28e44067e07d4501a840108f4dcf053c`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:08.649315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-64c8ed68-5c3f-470d-9856-b6b1c21dffbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:08.650231 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-65737b5e-c9ce-49a5-9908-4c81174e764e None None] MFA Rules not processed for user `dff34d763447487b9dfb46c0707b738b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:08.652882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-65737b5e-c9ce-49a5-9908-4c81174e764e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:08.747810 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 852/1711] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:08.747810 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8ed68-5c3f-470d-9856-b6b1c21dffbc None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:08.753512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-64c8ed68-5c3f-470d-9856-b6b1c21dffbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:08.845421 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 860/1712] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:08 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:08.845421 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65737b5e-c9ce-49a5-9908-4c81174e764e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:08.847291 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-65737b5e-c9ce-49a5-9908-4c81174e764e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:08.863439 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:09.035548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8ed68-5c3f-470d-9856-b6b1c21dffbc None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:09.050937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb3376-ae27-4e1c-9624-087c7b83b6be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:09.061086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb3376-ae27-4e1c-9624-087c7b83b6be service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:09.061086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb3376-ae27-4e1c-9624-087c7b83b6be service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:09.061086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb3376-ae27-4e1c-9624-087c7b83b6be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:09.061444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65737b5e-c9ce-49a5-9908-4c81174e764e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:09.070639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43240ddb-0d6c-4d0d-9f79-8576d064dffb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:09.070639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43240ddb-0d6c-4d0d-9f79-8576d064dffb service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:09.070639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43240ddb-0d6c-4d0d-9f79-8576d064dffb service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:09.070639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43240ddb-0d6c-4d0d-9f79-8576d064dffb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:09.086842 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43240ddb-0d6c-4d0d-9f79-8576d064dffb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:09.094735 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43240ddb-0d6c-4d0d-9f79-8576d064dffb service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:09.103622 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 861/1713] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:09.124376 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43240ddb-0d6c-4d0d-9f79-8576d064dffb service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:09.126943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:09.135621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-43240ddb-0d6c-4d0d-9f79-8576d064dffb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:09.173526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43240ddb-0d6c-4d0d-9f79-8576d064dffb service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:09.184337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dd6d0-3f0a-48a6-a7ea-c7af62a35904 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:09.184937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dd6d0-3f0a-48a6-a7ea-c7af62a35904 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:09.188659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dd6d0-3f0a-48a6-a7ea-c7af62a35904 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:09.189480 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dd6d0-3f0a-48a6-a7ea-c7af62a35904 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:09.206599 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dd6d0-3f0a-48a6-a7ea-c7af62a35904 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:09.210043 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dd6d0-3f0a-48a6-a7ea-c7af62a35904 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:09.245842 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 862/1714] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:09 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:09.281199 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb3376-ae27-4e1c-9624-087c7b83b6be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:09.285274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb3376-ae27-4e1c-9624-087c7b83b6be service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:09.331445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 853/1715] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:10.443453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dd6d0-3f0a-48a6-a7ea-c7af62a35904 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:10.491048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dd6d0-3f0a-48a6-a7ea-c7af62a35904 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:10.523806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45197d28-fcdd-4d74-8601-9bf13ca7ab51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:10.523806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45197d28-fcdd-4d74-8601-9bf13ca7ab51 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:10.523806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45197d28-fcdd-4d74-8601-9bf13ca7ab51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:10.523806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45197d28-fcdd-4d74-8601-9bf13ca7ab51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:10.696682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45197d28-fcdd-4d74-8601-9bf13ca7ab51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:10.699001 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45197d28-fcdd-4d74-8601-9bf13ca7ab51 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:10.722865 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 863/1716] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:10 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:11.636736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2723e9a-7d99-42a3-adda-ed7eb48c73c7 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:11.637330 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2723e9a-7d99-42a3-adda-ed7eb48c73c7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:11.637416 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2723e9a-7d99-42a3-adda-ed7eb48c73c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:11.714469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c2723e9a-7d99-42a3-adda-ed7eb48c73c7 None None] MFA Rules not processed for user `5d6b6321705947b6a395f77c11b1c6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:11.716517 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c2723e9a-7d99-42a3-adda-ed7eb48c73c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:11.856428 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:11.882453 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 854/1717] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:11.884751 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259b2a0-11f7-488f-955d-c6a95c879f58 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:11.885283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259b2a0-11f7-488f-955d-c6a95c879f58 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:11.885283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259b2a0-11f7-488f-955d-c6a95c879f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:11.910998 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b949c-a0a2-460b-b806-1a8e0e77d0e8 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:11.911440 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b949c-a0a2-460b-b806-1a8e0e77d0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:11.913136 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b949c-a0a2-460b-b806-1a8e0e77d0e8 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:11.915490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 855/1718] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 14 12:06:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:06:11.961383 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2259b2a0-11f7-488f-955d-c6a95c879f58 None None] MFA Rules not processed for user `5d6b6321705947b6a395f77c11b1c6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:11.966069 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2259b2a0-11f7-488f-955d-c6a95c879f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:11.985664 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 864/1719] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:11.998883 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b949c-a0a2-460b-b806-1a8e0e77d0e8 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:12.004759 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8c1b949c-a0a2-460b-b806-1a8e0e77d0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:12.017943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a16f38-9092-4cdf-b3b5-e65ee2011032 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:12.019353 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a16f38-9092-4cdf-b3b5-e65ee2011032 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:12.019722 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a16f38-9092-4cdf-b3b5-e65ee2011032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:12.086253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-80a16f38-9092-4cdf-b3b5-e65ee2011032 None None] MFA Rules not processed for user `0ca10dd7ed714edb8b87017deecc41b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:12.087736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-80a16f38-9092-4cdf-b3b5-e65ee2011032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:12.087866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b949c-a0a2-460b-b806-1a8e0e77d0e8 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:12.119845 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a3da9-d7f0-4529-aff6-cd2be9bbb979 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:12.122264 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a3da9-d7f0-4529-aff6-cd2be9bbb979 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:12.122551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a3da9-d7f0-4529-aff6-cd2be9bbb979 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:12.133117 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a3da9-d7f0-4529-aff6-cd2be9bbb979 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:12.194578 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a3da9-d7f0-4529-aff6-cd2be9bbb979 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:12.203774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a3da9-d7f0-4529-aff6-cd2be9bbb979 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:12.274545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 865/1720] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 14 12:06:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:06:12.276654 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fac60-07d6-4467-8476-808b141db6ef None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:12.277064 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fac60-07d6-4467-8476-808b141db6ef None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:12.277578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fac60-07d6-4467-8476-808b141db6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:12.283519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 856/1721] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:12.285055 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49930b2-1e29-4a89-84c9-22dc59b0908a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:12.285343 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49930b2-1e29-4a89-84c9-22dc59b0908a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:12.285613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49930b2-1e29-4a89-84c9-22dc59b0908a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:12.357211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:12.385084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2a7fac60-07d6-4467-8476-808b141db6ef None None] MFA Rules not processed for user `4bd4125ff6bb400bae1b94df7d2cfa17`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:12.386260 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2a7fac60-07d6-4467-8476-808b141db6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:12.410000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b49930b2-1e29-4a89-84c9-22dc59b0908a None None] MFA Rules not processed for user `2ba1860ce5504c27816a504776463d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:12.438379 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b49930b2-1e29-4a89-84c9-22dc59b0908a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:12.539310 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 866/1722] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:12.540215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fac60-07d6-4467-8476-808b141db6ef None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:12.599413 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fac60-07d6-4467-8476-808b141db6ef None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:12.638745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59784d-a0a0-4d05-a96b-5eb98110803c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca10dd7ed714edb8b87017deecc41b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:12.639392 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59784d-a0a0-4d05-a96b-5eb98110803c service placement] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:12.639678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59784d-a0a0-4d05-a96b-5eb98110803c service placement] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:12.639963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59784d-a0a0-4d05-a96b-5eb98110803c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:12.660691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59784d-a0a0-4d05-a96b-5eb98110803c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:12.662873 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59784d-a0a0-4d05-a96b-5eb98110803c service placement] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:12.702633 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 867/1723] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:12.707298 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59784d-a0a0-4d05-a96b-5eb98110803c service placement] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:12.710440 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7b59784d-a0a0-4d05-a96b-5eb98110803c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:12.751843 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 857/1724] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:12.753822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab706c-6a7a-4881-aa3f-84a345f8584f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:12.754169 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab706c-6a7a-4881-aa3f-84a345f8584f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:12.754443 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab706c-6a7a-4881-aa3f-84a345f8584f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:12.807321 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-2eab706c-6a7a-4881-aa3f-84a345f8584f None None] MFA Rules not processed for user `4bd4125ff6bb400bae1b94df7d2cfa17`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:12.810049 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2eab706c-6a7a-4881-aa3f-84a345f8584f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:12.832753 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 858/1725] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:12.837678 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab706c-6a7a-4881-aa3f-84a345f8584f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:12.868051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab706c-6a7a-4881-aa3f-84a345f8584f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:12.877785 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e3c9d-1d25-459f-b086-fd268354681d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:12.878891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e3c9d-1d25-459f-b086-fd268354681d service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:12.884255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e3c9d-1d25-459f-b086-fd268354681d service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:12.884833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e3c9d-1d25-459f-b086-fd268354681d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:12.897214 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e3c9d-1d25-459f-b086-fd268354681d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:12.899976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e3c9d-1d25-459f-b086-fd268354681d service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:12.915656 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 859/1726] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:12.956249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59784d-a0a0-4d05-a96b-5eb98110803c service placement] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:12.982887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88192c-3cbc-449e-a364-ada98a3b45a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:12.985041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88192c-3cbc-449e-a364-ada98a3b45a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:12.985681 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88192c-3cbc-449e-a364-ada98a3b45a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:12.986129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88192c-3cbc-449e-a364-ada98a3b45a9 admin admin] PATH_INFO: `/v3/users/365d435bfe3a4c00956382364fb47edc` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:12.987948 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88192c-3cbc-449e-a364-ada98a3b45a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=365d435bfe3a4c00956382364fb47edc)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:13.014116 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88192c-3cbc-449e-a364-ada98a3b45a9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:13.130788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e3c9d-1d25-459f-b086-fd268354681d service nova] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:13.149220 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e3c9d-1d25-459f-b086-fd268354681d service nova] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:13.162952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d6383-9c6c-4ea0-a309-aa65c944631c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:13.169247 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d6383-9c6c-4ea0-a309-aa65c944631c service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:13.169247 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d6383-9c6c-4ea0-a309-aa65c944631c service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:13.169247 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d6383-9c6c-4ea0-a309-aa65c944631c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:13.192261 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d6383-9c6c-4ea0-a309-aa65c944631c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:13.194235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d6383-9c6c-4ea0-a309-aa65c944631c service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:13.235606 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 860/1727] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:13 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:13.471139 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3d88192c-3cbc-449e-a364-ada98a3b45a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '365d435bfe3a4c00956382364fb47edc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:13.511667 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225e7cb-8247-4789-a828-c4edd1096f6b None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:13.511953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225e7cb-8247-4789-a828-c4edd1096f6b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:13.512202 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225e7cb-8247-4789-a828-c4edd1096f6b None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:13.513944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 861/1728] 10.222.0.62 () {54 vars in 912 bytes} [Wed Sep 14 12:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 12:06:13.521963 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadfae4-473b-4ede-b60f-491b2e22e45c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:13.522314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadfae4-473b-4ede-b60f-491b2e22e45c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:13.522499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadfae4-473b-4ede-b60f-491b2e22e45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:13.560592 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3d88192c-3cbc-449e-a364-ada98a3b45a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '365d435bfe3a4c00956382364fb47edc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:13.587651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3d88192c-3cbc-449e-a364-ada98a3b45a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '365d435bfe3a4c00956382364fb47edc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:13.653174 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ddadfae4-473b-4ede-b60f-491b2e22e45c None None] MFA Rules not processed for user `13f1840d92d94c3ebd16000a4889c377`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:13.654698 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ddadfae4-473b-4ede-b60f-491b2e22e45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:13.739178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3d88192c-3cbc-449e-a364-ada98a3b45a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '365d435bfe3a4c00956382364fb47edc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:13.742996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3d88192c-3cbc-449e-a364-ada98a3b45a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '365d435bfe3a4c00956382364fb47edc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:13.749422 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 868/1729] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:06:12 2022] DELETE /identity/v3/users/365d435bfe3a4c00956382364fb47edc => generated 0 bytes in 1046 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:13.810857 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 862/1730] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 14 12:06:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:06:13.851612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88192c-3cbc-449e-a364-ada98a3b45a9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:13.853400 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3d88192c-3cbc-449e-a364-ada98a3b45a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:13.868352 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:13.991514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88192c-3cbc-449e-a364-ada98a3b45a9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:14.001130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d070d4-86cb-42c4-bfd7-82ea81c4a12c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:14.001780 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d070d4-86cb-42c4-bfd7-82ea81c4a12c service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:14.002093 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d070d4-86cb-42c4-bfd7-82ea81c4a12c service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:14.002404 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d070d4-86cb-42c4-bfd7-82ea81c4a12c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:14.018776 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d070d4-86cb-42c4-bfd7-82ea81c4a12c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:14.020337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d070d4-86cb-42c4-bfd7-82ea81c4a12c service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:14.045021 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 869/1731] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:14.295543 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c4abf-7949-4ce1-bb2f-0299982bd725 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:14.296159 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c4abf-7949-4ce1-bb2f-0299982bd725 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:14.296694 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c4abf-7949-4ce1-bb2f-0299982bd725 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:14.298467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 863/1732] 10.222.0.62 () {54 vars in 912 bytes} [Wed Sep 14 12:06:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:06:14.305780 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458a98a-1ee9-43b7-a9d3-c0bc19b0a947 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:14.306642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458a98a-1ee9-43b7-a9d3-c0bc19b0a947 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:14.307033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458a98a-1ee9-43b7-a9d3-c0bc19b0a947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:14.350237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1458a98a-1ee9-43b7-a9d3-c0bc19b0a947 None None] MFA Rules not processed for user `13f1840d92d94c3ebd16000a4889c377`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:14.351524 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1458a98a-1ee9-43b7-a9d3-c0bc19b0a947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:14.368594 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 870/1733] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 14 12:06:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:14.409554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c4abf-7949-4ce1-bb2f-0299982bd725 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:14.429534 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c4abf-7949-4ce1-bb2f-0299982bd725 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:14.436663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af362e9e-3bbf-4007-837e-c43114873de1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:14.437474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af362e9e-3bbf-4007-837e-c43114873de1 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:14.437810 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af362e9e-3bbf-4007-837e-c43114873de1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:14.438094 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af362e9e-3bbf-4007-837e-c43114873de1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:14.463349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af362e9e-3bbf-4007-837e-c43114873de1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:14.467298 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af362e9e-3bbf-4007-837e-c43114873de1 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:14.495654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 864/1734] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:14.741358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458a98a-1ee9-43b7-a9d3-c0bc19b0a947 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:14.743586 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1458a98a-1ee9-43b7-a9d3-c0bc19b0a947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:14.799142 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfb9da-3968-440c-9eed-811cc163361c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:14.799444 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfb9da-3968-440c-9eed-811cc163361c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:14.799919 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfb9da-3968-440c-9eed-811cc163361c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:14.850223 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-06cfb9da-3968-440c-9eed-811cc163361c None None] MFA Rules not processed for user `28e44067e07d4501a840108f4dcf053c`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:14.852458 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-06cfb9da-3968-440c-9eed-811cc163361c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:14.928851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:14.938405 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458a98a-1ee9-43b7-a9d3-c0bc19b0a947 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:14.956212 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b5bf6-5bdc-43c1-9140-be253b361f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:14.956778 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b5bf6-5bdc-43c1-9140-be253b361f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:14.957564 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b5bf6-5bdc-43c1-9140-be253b361f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:14.957564 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b5bf6-5bdc-43c1-9140-be253b361f95 admin admin] PATH_INFO: `/v3/projects/5493cbcbaefb44f192bdffa1b3f6fd05` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:14.971666 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b5bf6-5bdc-43c1-9140-be253b361f95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5493cbcbaefb44f192bdffa1b3f6fd05)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:14.973555 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b5bf6-5bdc-43c1-9140-be253b361f95 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:15.028235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 865/1735] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:06:15.109772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfb9da-3968-440c-9eed-811cc163361c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:15.112385 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-06cfb9da-3968-440c-9eed-811cc163361c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:15.186610 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c02b5bf6-5bdc-43c1-9140-be253b361f95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5493cbcbaefb44f192bdffa1b3f6fd05', 'request_id': 'req-c02b5bf6-5bdc-43c1-9140-be253b361f95'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:15.281956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c02b5bf6-5bdc-43c1-9140-be253b361f95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5493cbcbaefb44f192bdffa1b3f6fd05', 'request_id': 'req-c02b5bf6-5bdc-43c1-9140-be253b361f95'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:15.292389 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c02b5bf6-5bdc-43c1-9140-be253b361f95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5493cbcbaefb44f192bdffa1b3f6fd05', 'request_id': 'req-c02b5bf6-5bdc-43c1-9140-be253b361f95'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:15.333227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfb9da-3968-440c-9eed-811cc163361c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:15.344836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c02b5bf6-5bdc-43c1-9140-be253b361f95 admin admin] The token cache is being invalidate because project 5493cbcbaefb44f192bdffa1b3f6fd05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:06:15.345754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c02b5bf6-5bdc-43c1-9140-be253b361f95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:15.350707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c02b5bf6-5bdc-43c1-9140-be253b361f95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:15.360562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 871/1736] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:06:14 2022] DELETE /identity/v3/projects/5493cbcbaefb44f192bdffa1b3f6fd05 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:15.363869 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b5bf6-5bdc-43c1-9140-be253b361f95 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:15.374700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c02b5bf6-5bdc-43c1-9140-be253b361f95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:15.490462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:15.536771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc196d-4048-49f5-8619-eb04a31bdf19 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca10dd7ed714edb8b87017deecc41b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:15.539564 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc196d-4048-49f5-8619-eb04a31bdf19 service placement] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:15.540122 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc196d-4048-49f5-8619-eb04a31bdf19 service placement] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:15.540645 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc196d-4048-49f5-8619-eb04a31bdf19 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:15.543446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b0fc196d-4048-49f5-8619-eb04a31bdf19 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:15.579395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc196d-4048-49f5-8619-eb04a31bdf19 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:15.579395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc196d-4048-49f5-8619-eb04a31bdf19 service placement] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:15.590781 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 866/1737] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:06:15.590781 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66deb8-29bd-4c40-b253-63fe096d8a67 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:15.600496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b5bf6-5bdc-43c1-9140-be253b361f95 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:15.600707 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66deb8-29bd-4c40-b253-63fe096d8a67 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:15.600707 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66deb8-29bd-4c40-b253-63fe096d8a67 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:15.608739 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 867/1738] 10.222.0.62 () {54 vars in 961 bytes} [Wed Sep 14 12:06:15 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:06:15.610989 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495cca3-c38b-4e03-91ac-808bcaae3dc9 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:15.623228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495cca3-c38b-4e03-91ac-808bcaae3dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:15.623228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495cca3-c38b-4e03-91ac-808bcaae3dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:15.623864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2746c-7354-4972-abca-8a7b4c5893bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:15.624786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2746c-7354-4972-abca-8a7b4c5893bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:15.625786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2746c-7354-4972-abca-8a7b4c5893bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:15.626324 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2746c-7354-4972-abca-8a7b4c5893bb admin admin] PATH_INFO: `/v3/users/d9d8955f8bdb499eab93afb503505ccd` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:15.628662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2746c-7354-4972-abca-8a7b4c5893bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9d8955f8bdb499eab93afb503505ccd)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:15.631053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2746c-7354-4972-abca-8a7b4c5893bb admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:15.648193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c495cca3-c38b-4e03-91ac-808bcaae3dc9 None None] MFA Rules not processed for user `0ca10dd7ed714edb8b87017deecc41b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:15.652991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c495cca3-c38b-4e03-91ac-808bcaae3dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:15.679228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 868/1739] 10.222.0.62 () {58 vars in 1058 bytes} [Wed Sep 14 12:06:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 14 12:06:15.680535 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139e76e-a110-46c5-99c8-4fb69c05a3c9 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:15.680827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139e76e-a110-46c5-99c8-4fb69c05a3c9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:15.681183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139e76e-a110-46c5-99c8-4fb69c05a3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:15.706296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3139e76e-a110-46c5-99c8-4fb69c05a3c9 None None] MFA Rules not processed for user `4bd4125ff6bb400bae1b94df7d2cfa17`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:15.708401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3139e76e-a110-46c5-99c8-4fb69c05a3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:15.836531 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:15.863709 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 869/1740] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:15.864594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139e76e-a110-46c5-99c8-4fb69c05a3c9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:15.902932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139e76e-a110-46c5-99c8-4fb69c05a3c9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:15.924873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777ddb1-2510-4a33-9ef9-a134c6c5363f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca10dd7ed714edb8b87017deecc41b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:15.928732 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777ddb1-2510-4a33-9ef9-a134c6c5363f service placement] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:15.929099 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777ddb1-2510-4a33-9ef9-a134c6c5363f service placement] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:15.930569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-82c2746c-7354-4972-abca-8a7b4c5893bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9d8955f8bdb499eab93afb503505ccd'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:15.936506 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777ddb1-2510-4a33-9ef9-a134c6c5363f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:15.950536 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-82c2746c-7354-4972-abca-8a7b4c5893bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9d8955f8bdb499eab93afb503505ccd'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:15.964816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-82c2746c-7354-4972-abca-8a7b4c5893bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9d8955f8bdb499eab93afb503505ccd'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:15.977938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777ddb1-2510-4a33-9ef9-a134c6c5363f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:15.981622 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777ddb1-2510-4a33-9ef9-a134c6c5363f service placement] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:16.031031 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 870/1741] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:16.059099 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-82c2746c-7354-4972-abca-8a7b4c5893bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9d8955f8bdb499eab93afb503505ccd'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:16.060034 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-82c2746c-7354-4972-abca-8a7b4c5893bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9d8955f8bdb499eab93afb503505ccd'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:16.070166 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 872/1742] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:06:15 2022] DELETE /identity/v3/users/d9d8955f8bdb499eab93afb503505ccd => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:16.612970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777ddb1-2510-4a33-9ef9-a134c6c5363f service placement] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:16.628479 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777ddb1-2510-4a33-9ef9-a134c6c5363f service placement] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:16.749402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d1f31-85e7-4572-9a2e-15c8d9450a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:16.752987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d1f31-85e7-4572-9a2e-15c8d9450a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:16.752987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d1f31-85e7-4572-9a2e-15c8d9450a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:16.752987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d1f31-85e7-4572-9a2e-15c8d9450a08 admin admin] PATH_INFO: `/v3/projects/660c8ff838b2437093a7823701be08ec` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:16.752987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d1f31-85e7-4572-9a2e-15c8d9450a08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=660c8ff838b2437093a7823701be08ec)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:16.754922 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d1f31-85e7-4572-9a2e-15c8d9450a08 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:16.961052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-876d1f31-85e7-4572-9a2e-15c8d9450a08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '660c8ff838b2437093a7823701be08ec', 'request_id': 'req-876d1f31-85e7-4572-9a2e-15c8d9450a08'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:17.049848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-876d1f31-85e7-4572-9a2e-15c8d9450a08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '660c8ff838b2437093a7823701be08ec', 'request_id': 'req-876d1f31-85e7-4572-9a2e-15c8d9450a08'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:17.067671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-876d1f31-85e7-4572-9a2e-15c8d9450a08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '660c8ff838b2437093a7823701be08ec', 'request_id': 'req-876d1f31-85e7-4572-9a2e-15c8d9450a08'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:17.088804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-876d1f31-85e7-4572-9a2e-15c8d9450a08 admin admin] The token cache is being invalidate because project 660c8ff838b2437093a7823701be08ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:06:17.089035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-876d1f31-85e7-4572-9a2e-15c8d9450a08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:17.089894 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-876d1f31-85e7-4572-9a2e-15c8d9450a08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:17.091623 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 871/1743] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:06:16 2022] DELETE /identity/v3/projects/660c8ff838b2437093a7823701be08ec => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:17.097337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74df199-5b23-4973-a2e1-354cc42eebc9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:17.098963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74df199-5b23-4973-a2e1-354cc42eebc9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:17.099669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74df199-5b23-4973-a2e1-354cc42eebc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:17.120952 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-c74df199-5b23-4973-a2e1-354cc42eebc9 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:17.122958 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c74df199-5b23-4973-a2e1-354cc42eebc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:17.297907 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e4ce4-23ce-45eb-80a1-777595a7cda9 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:17.303437 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e4ce4-23ce-45eb-80a1-777595a7cda9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:17.303437 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e4ce4-23ce-45eb-80a1-777595a7cda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:17.303773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 873/1744] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:17.329229 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-210e4ce4-23ce-45eb-80a1-777595a7cda9 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:17.331833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-210e4ce4-23ce-45eb-80a1-777595a7cda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:17.343327 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 872/1745] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:17.348490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74df199-5b23-4973-a2e1-354cc42eebc9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:17.362903 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74df199-5b23-4973-a2e1-354cc42eebc9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:17.369681 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cacfc-65f3-4bab-b017-182321036820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:17.370371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cacfc-65f3-4bab-b017-182321036820 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:17.370648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cacfc-65f3-4bab-b017-182321036820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:17.371154 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cacfc-65f3-4bab-b017-182321036820 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:17.371913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513cacfc-65f3-4bab-b017-182321036820 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:06:17.372237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513cacfc-65f3-4bab-b017-182321036820 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:17.373962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513cacfc-65f3-4bab-b017-182321036820 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:17.391907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 874/1746] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:06:17.392365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e4ce4-23ce-45eb-80a1-777595a7cda9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:17.404255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e4ce4-23ce-45eb-80a1-777595a7cda9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:17.410522 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e912e80f-18cc-4c8b-8bc6-b54ee16b51bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:17.411153 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912e80f-18cc-4c8b-8bc6-b54ee16b51bc admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:17.411621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912e80f-18cc-4c8b-8bc6-b54ee16b51bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:17.413198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912e80f-18cc-4c8b-8bc6-b54ee16b51bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:17.413198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e912e80f-18cc-4c8b-8bc6-b54ee16b51bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:17.414614 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e912e80f-18cc-4c8b-8bc6-b54ee16b51bc admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:17.524189 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 873/1747] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:06:17 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:06:17.528831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cacfc-65f3-4bab-b017-182321036820 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:17.549610 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cacfc-65f3-4bab-b017-182321036820 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:17.686601 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:17.729879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb11f4-f8a0-44c8-b397-73dfa425265e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:17.729879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb11f4-f8a0-44c8-b397-73dfa425265e admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:17.729879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb11f4-f8a0-44c8-b397-73dfa425265e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:17.729879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb11f4-f8a0-44c8-b397-73dfa425265e admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:17.730233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb11f4-f8a0-44c8-b397-73dfa425265e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:17.734466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb11f4-f8a0-44c8-b397-73dfa425265e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:17.849271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 875/1748] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:06:17 2022] POST /identity/v3/users => generated 291 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:06:17.853287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e912e80f-18cc-4c8b-8bc6-b54ee16b51bc admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:17.872663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e912e80f-18cc-4c8b-8bc6-b54ee16b51bc admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:17.884151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bde79-1754-4a5b-bce3-24a73a449511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:17.893496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bde79-1754-4a5b-bce3-24a73a449511 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:17.893496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bde79-1754-4a5b-bce3-24a73a449511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:17.893496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bde79-1754-4a5b-bce3-24a73a449511 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:17.893496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bde79-1754-4a5b-bce3-24a73a449511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:17.893496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bde79-1754-4a5b-bce3-24a73a449511 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:17.909996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 874/1749] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:06:17.912539 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb11f4-f8a0-44c8-b397-73dfa425265e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:17.933548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb11f4-f8a0-44c8-b397-73dfa425265e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:17.942695 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444d7d0-9e70-44d2-9111-4984d641d0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:17.943423 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444d7d0-9e70-44d2-9111-4984d641d0f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:17.943736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444d7d0-9e70-44d2-9111-4984d641d0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:17.944017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444d7d0-9e70-44d2-9111-4984d641d0f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:17.944688 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444d7d0-9e70-44d2-9111-4984d641d0f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:17.946345 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444d7d0-9e70-44d2-9111-4984d641d0f4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:17.966043 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 876/1750] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:06:17.970784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bde79-1754-4a5b-bce3-24a73a449511 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:17.996480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bde79-1754-4a5b-bce3-24a73a449511 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:17.996480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b6f7d-d468-4f41-83f7-2210404d23ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:17.996480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b6f7d-d468-4f41-83f7-2210404d23ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:17.998237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b6f7d-d468-4f41-83f7-2210404d23ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:17.999497 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b6f7d-d468-4f41-83f7-2210404d23ee admin admin] PATH_INFO: `/v3/projects/1b87db87166c4c58ac35e2b66ed7bc7e/users/98783397ee9a4d799a5541f447287da2/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:18.063276 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b6f7d-d468-4f41-83f7-2210404d23ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b87db87166c4c58ac35e2b66ed7bc7e, user_id=98783397ee9a4d799a5541f447287da2, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:18.077060 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b6f7d-d468-4f41-83f7-2210404d23ee admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:18.244526 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 875/1751] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:06:17 2022] PUT /identity/v3/projects/1b87db87166c4c58ac35e2b66ed7bc7e/users/98783397ee9a4d799a5541f447287da2/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:06:18.256802 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9170cb8-070f-404f-9652-2bda3b48bf92 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:18.258116 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9170cb8-070f-404f-9652-2bda3b48bf92 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:18.258597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9170cb8-070f-404f-9652-2bda3b48bf92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:18.335624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f9170cb8-070f-404f-9652-2bda3b48bf92 None None] MFA Rules not processed for user `98783397ee9a4d799a5541f447287da2`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:18.337002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f9170cb8-070f-404f-9652-2bda3b48bf92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:18.434040 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 877/1752] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:18.442413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b6f7d-d468-4f41-83f7-2210404d23ee admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:18.457148 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:18.517211 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b6f7d-d468-4f41-83f7-2210404d23ee admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:18.526124 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31430545-1464-40c2-b62e-f8f32735eefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:18.526969 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31430545-1464-40c2-b62e-f8f32735eefe admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:18.527488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31430545-1464-40c2-b62e-f8f32735eefe admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:18.527923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31430545-1464-40c2-b62e-f8f32735eefe admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:18.528672 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31430545-1464-40c2-b62e-f8f32735eefe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:18.536605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31430545-1464-40c2-b62e-f8f32735eefe admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:18.709170 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 876/1753] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:06:18 2022] POST /identity/v3/projects => generated 381 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:06:18.711501 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9170cb8-070f-404f-9652-2bda3b48bf92 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:18.745478 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9170cb8-070f-404f-9652-2bda3b48bf92 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:18.841365 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7089ea3-9aa5-4008-b8f2-57e32922adce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:18.842023 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7089ea3-9aa5-4008-b8f2-57e32922adce admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:18.842309 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7089ea3-9aa5-4008-b8f2-57e32922adce admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:18.842584 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7089ea3-9aa5-4008-b8f2-57e32922adce admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:18.843218 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7089ea3-9aa5-4008-b8f2-57e32922adce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:18.845140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7089ea3-9aa5-4008-b8f2-57e32922adce admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:18.913306 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d96180-3c07-47b7-b33e-55249085aa6e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:18.913597 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d96180-3c07-47b7-b33e-55249085aa6e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:18.913946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d96180-3c07-47b7-b33e-55249085aa6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:18.959784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 878/1754] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:06:18 2022] POST /identity/v3/users => generated 297 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:06:18.962040 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7089ea3-9aa5-4008-b8f2-57e32922adce admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:18.968963 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-99d96180-3c07-47b7-b33e-55249085aa6e None None] MFA Rules not processed for user `2ba1860ce5504c27816a504776463d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:18.970236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-99d96180-3c07-47b7-b33e-55249085aa6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:18.975475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7089ea3-9aa5-4008-b8f2-57e32922adce admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:18.980744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d32aa2-9403-4dc4-ac00-d2bd7807cdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:18.981547 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d32aa2-9403-4dc4-ac00-d2bd7807cdcd admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:18.982043 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d32aa2-9403-4dc4-ac00-d2bd7807cdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:18.982473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d32aa2-9403-4dc4-ac00-d2bd7807cdcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:18.983640 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d32aa2-9403-4dc4-ac00-d2bd7807cdcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:18.985675 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d32aa2-9403-4dc4-ac00-d2bd7807cdcd admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:19.036263 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 879/1755] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:06:19.040947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d32aa2-9403-4dc4-ac00-d2bd7807cdcd admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:19.057034 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d32aa2-9403-4dc4-ac00-d2bd7807cdcd admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:19.064565 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a93b0-53c2-4541-ae78-ed3116019ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:19.068279 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a93b0-53c2-4541-ae78-ed3116019ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:19.068935 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a93b0-53c2-4541-ae78-ed3116019ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:19.069497 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a93b0-53c2-4541-ae78-ed3116019ad4 admin admin] PATH_INFO: `/v3/projects/aa46bc3c534f4d4bbe7db6c3b315b5f9/users/0d9abac08970410992485358655cde87/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:19.088945 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a93b0-53c2-4541-ae78-ed3116019ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa46bc3c534f4d4bbe7db6c3b315b5f9, user_id=0d9abac08970410992485358655cde87, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:19.092457 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a93b0-53c2-4541-ae78-ed3116019ad4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:19.095262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 877/1756] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:19.213342 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:19.218711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 880/1757] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:06:19 2022] PUT /identity/v3/projects/aa46bc3c534f4d4bbe7db6c3b315b5f9/users/0d9abac08970410992485358655cde87/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:19.223458 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d96180-3c07-47b7-b33e-55249085aa6e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:19.245172 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d96180-3c07-47b7-b33e-55249085aa6e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:19.319559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba011f9-0f09-4b14-a9dd-dce81fbae9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:19.320661 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba011f9-0f09-4b14-a9dd-dce81fbae9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:19.321147 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba011f9-0f09-4b14-a9dd-dce81fbae9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:19.322421 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba011f9-0f09-4b14-a9dd-dce81fbae9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:19.323528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba011f9-0f09-4b14-a9dd-dce81fbae9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:19.325560 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba011f9-0f09-4b14-a9dd-dce81fbae9a6 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:19.343208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 878/1758] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:06:19.348925 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a93b0-53c2-4541-ae78-ed3116019ad4 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:19.363974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a93b0-53c2-4541-ae78-ed3116019ad4 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:19.369726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460ea6c-aabe-4ad5-a706-13ac9f0521b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:19.370513 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460ea6c-aabe-4ad5-a706-13ac9f0521b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:19.370792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460ea6c-aabe-4ad5-a706-13ac9f0521b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:19.371072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460ea6c-aabe-4ad5-a706-13ac9f0521b2 admin admin] PATH_INFO: `/v3/projects/aa46bc3c534f4d4bbe7db6c3b315b5f9/users/0d9abac08970410992485358655cde87/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:19.373210 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460ea6c-aabe-4ad5-a706-13ac9f0521b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa46bc3c534f4d4bbe7db6c3b315b5f9, user_id=0d9abac08970410992485358655cde87, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:19.376048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460ea6c-aabe-4ad5-a706-13ac9f0521b2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:19.394009 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 881/1759] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:06:19 2022] PUT /identity/v3/projects/aa46bc3c534f4d4bbe7db6c3b315b5f9/users/0d9abac08970410992485358655cde87/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:19.407763 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e8208-8e19-4ba7-96c2-1cb4720e1874 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:19.408052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e8208-8e19-4ba7-96c2-1cb4720e1874 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:19.408359 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e8208-8e19-4ba7-96c2-1cb4720e1874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:19.452830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4c7e8208-8e19-4ba7-96c2-1cb4720e1874 None None] MFA Rules not processed for user `0d9abac08970410992485358655cde87`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:19.454294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4c7e8208-8e19-4ba7-96c2-1cb4720e1874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:19.518944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:19.583916 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 879/1760] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:19 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:19.595135 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460ea6c-aabe-4ad5-a706-13ac9f0521b2 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:19.602420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8460ea6c-aabe-4ad5-a706-13ac9f0521b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:19.693307 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460ea6c-aabe-4ad5-a706-13ac9f0521b2 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:19.698443 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53eadf9-be8c-4507-9427-830f2b585022 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:19.699280 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53eadf9-be8c-4507-9427-830f2b585022 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:19.699562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53eadf9-be8c-4507-9427-830f2b585022 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:19.699848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53eadf9-be8c-4507-9427-830f2b585022 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:19.711986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53eadf9-be8c-4507-9427-830f2b585022 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:19.713558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53eadf9-be8c-4507-9427-830f2b585022 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:19.725543 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 882/1761] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:19.981210 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e8208-8e19-4ba7-96c2-1cb4720e1874 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:19.989411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e8208-8e19-4ba7-96c2-1cb4720e1874 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:19.993418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad596a-e9ba-4893-8eb0-73d172960498 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:19.994073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad596a-e9ba-4893-8eb0-73d172960498 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:19.994383 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad596a-e9ba-4893-8eb0-73d172960498 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:19.994752 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad596a-e9ba-4893-8eb0-73d172960498 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:20.060070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad596a-e9ba-4893-8eb0-73d172960498 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:20.061518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad596a-e9ba-4893-8eb0-73d172960498 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:20.071906 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 880/1762] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:24.305339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53eadf9-be8c-4507-9427-830f2b585022 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:24.314632 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53eadf9-be8c-4507-9427-830f2b585022 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:24.322520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a84f1-41e3-400e-9107-cda9671cc75b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:24.322520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a84f1-41e3-400e-9107-cda9671cc75b service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:24.322520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a84f1-41e3-400e-9107-cda9671cc75b service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:24.322520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a84f1-41e3-400e-9107-cda9671cc75b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:24.324597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-bf1a84f1-41e3-400e-9107-cda9671cc75b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:24.395249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1a84f1-41e3-400e-9107-cda9671cc75b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:24.396599 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1a84f1-41e3-400e-9107-cda9671cc75b service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:24.407124 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 883/1763] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:06:24.604726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962af92-0a65-408f-8f9e-04eb42fb1444 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:24.605235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962af92-0a65-408f-8f9e-04eb42fb1444 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:24.605346 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962af92-0a65-408f-8f9e-04eb42fb1444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:24.733066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f962af92-0a65-408f-8f9e-04eb42fb1444 None None] MFA Rules not processed for user `d76a164ed7e14d9eb2db016b68a84d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:24.734928 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f962af92-0a65-408f-8f9e-04eb42fb1444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:24.866356 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 881/1764] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 14 12:06:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:25.253795 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a84f1-41e3-400e-9107-cda9671cc75b service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:25.255361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-bf1a84f1-41e3-400e-9107-cda9671cc75b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:25.527572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a84f1-41e3-400e-9107-cda9671cc75b service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:25.538857 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feacda5b-4c77-4553-93aa-86811225f931 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1047c7d18c1466d989cd895117cd807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:25.538857 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feacda5b-4c77-4553-93aa-86811225f931 service swift] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:25.538857 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feacda5b-4c77-4553-93aa-86811225f931 service swift] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:25.538857 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feacda5b-4c77-4553-93aa-86811225f931 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:25.546554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:25.554929 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feacda5b-4c77-4553-93aa-86811225f931 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:25.556736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feacda5b-4c77-4553-93aa-86811225f931 service swift] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:25.571032 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 884/1765] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 14 12:06:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 12:06:34.852850 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=84 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:36.458830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:40.614269 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74901996-0e7b-4572-95a3-c91668761636 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:40.614910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74901996-0e7b-4572-95a3-c91668761636 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:40.614910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74901996-0e7b-4572-95a3-c91668761636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:40.636927 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-74901996-0e7b-4572-95a3-c91668761636 None None] MFA Rules not processed for user `13f1840d92d94c3ebd16000a4889c377`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:40.638193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-74901996-0e7b-4572-95a3-c91668761636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:40.716202 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 882/1766] 10.222.0.81 () {58 vars in 1015 bytes} [Wed Sep 14 12:06:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:40.734458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feacda5b-4c77-4553-93aa-86811225f931 service swift] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:40.738781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-feacda5b-4c77-4553-93aa-86811225f931 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:40.757647 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74901996-0e7b-4572-95a3-c91668761636 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:40.758161 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feacda5b-4c77-4553-93aa-86811225f931 service swift] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:40.762172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe1376-19ab-4024-9c3a-7ab3f392ca6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:40.768259 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe1376-19ab-4024-9c3a-7ab3f392ca6e service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:40.768577 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-74901996-0e7b-4572-95a3-c91668761636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:40.768684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe1376-19ab-4024-9c3a-7ab3f392ca6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:40.768684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe1376-19ab-4024-9c3a-7ab3f392ca6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:40.772856 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe1376-19ab-4024-9c3a-7ab3f392ca6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:40.774462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe1376-19ab-4024-9c3a-7ab3f392ca6e service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:40.774820 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74901996-0e7b-4572-95a3-c91668761636 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:40.780754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf790c-1a15-4a6a-b99d-98691b7c9214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:40.781697 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf790c-1a15-4a6a-b99d-98691b7c9214 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:40.782076 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf790c-1a15-4a6a-b99d-98691b7c9214 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:40.782419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf790c-1a15-4a6a-b99d-98691b7c9214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:40.796957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf790c-1a15-4a6a-b99d-98691b7c9214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:40.796957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf790c-1a15-4a6a-b99d-98691b7c9214 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:40.801399 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 885/1767] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:40.807759 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 883/1768] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:06:42.254213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b72f5-3476-4192-882f-96f0054ffbd8 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:42.254736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b72f5-3476-4192-882f-96f0054ffbd8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:42.254824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b72f5-3476-4192-882f-96f0054ffbd8 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:42.256286 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 886/1769] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 14 12:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:06:42.276627 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5432d3-716c-48a0-bced-c32cdf13f7fc None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:42.280630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5432d3-716c-48a0-bced-c32cdf13f7fc None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:42.280630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5432d3-716c-48a0-bced-c32cdf13f7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:42.323961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-2c5432d3-716c-48a0-bced-c32cdf13f7fc None None] MFA Rules not processed for user `38b093bc4c1d4ebcbd2a6bb16b70c029`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:42.325384 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2c5432d3-716c-48a0-bced-c32cdf13f7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:42.465264 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 884/1770] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 14 12:06:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:42.604983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b72f5-3476-4192-882f-96f0054ffbd8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:42.604983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ef0b72f5-3476-4192-882f-96f0054ffbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:42.639568 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b72f5-3476-4192-882f-96f0054ffbd8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:42.649147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c71d6a-305d-4d0b-b061-4c3a83b39979 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:42.649834 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c71d6a-305d-4d0b-b061-4c3a83b39979 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:42.650536 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c71d6a-305d-4d0b-b061-4c3a83b39979 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:42.656092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c71d6a-305d-4d0b-b061-4c3a83b39979 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:42.684832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c71d6a-305d-4d0b-b061-4c3a83b39979 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:42.691621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c71d6a-305d-4d0b-b061-4c3a83b39979 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:42.696464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779afca9-c2a3-4deb-84cc-81fada792460 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:42.697401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779afca9-c2a3-4deb-84cc-81fada792460 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:42.698146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779afca9-c2a3-4deb-84cc-81fada792460 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:42.703294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 885/1771] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 14 12:06:42 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 12:06:42.724470 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 887/1772] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:42.744648 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf21e7e-f18e-4619-9464-f41c77fb70a9 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:42.745797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf21e7e-f18e-4619-9464-f41c77fb70a9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:42.746298 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf21e7e-f18e-4619-9464-f41c77fb70a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:42.797001 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-aaf21e7e-f18e-4619-9464-f41c77fb70a9 None None] MFA Rules not processed for user `38b093bc4c1d4ebcbd2a6bb16b70c029`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:42.798813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-aaf21e7e-f18e-4619-9464-f41c77fb70a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:42.810426 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 886/1773] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 14 12:06:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:42.937699 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c71d6a-305d-4d0b-b061-4c3a83b39979 service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:42.952352 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c71d6a-305d-4d0b-b061-4c3a83b39979 service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:42.972730 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ccadcd-7d27-4758-8952-216884c456ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:42.973473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccadcd-7d27-4758-8952-216884c456ea service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:42.973887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccadcd-7d27-4758-8952-216884c456ea service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:42.974298 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccadcd-7d27-4758-8952-216884c456ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:43.002506 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ccadcd-7d27-4758-8952-216884c456ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:43.004828 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ccadcd-7d27-4758-8952-216884c456ea service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:43.046176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 888/1774] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:52.784602 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=63 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:53.019500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=18 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:53.240378 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf21e7e-f18e-4619-9464-f41c77fb70a9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:53.245357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-aaf21e7e-f18e-4619-9464-f41c77fb70a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:53.325469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf21e7e-f18e-4619-9464-f41c77fb70a9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:53.330945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd775bfe-ac1f-4581-a25a-ed4a95abb783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:53.331927 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd775bfe-ac1f-4581-a25a-ed4a95abb783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:53.332215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd775bfe-ac1f-4581-a25a-ed4a95abb783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:53.332504 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd775bfe-ac1f-4581-a25a-ed4a95abb783 admin admin] PATH_INFO: `/v3/users/f04baf2739c0430e8514faa2c42de104` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:53.333780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd775bfe-ac1f-4581-a25a-ed4a95abb783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f04baf2739c0430e8514faa2c42de104)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:53.335804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd775bfe-ac1f-4581-a25a-ed4a95abb783 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:53.599735 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-dd775bfe-ac1f-4581-a25a-ed4a95abb783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f04baf2739c0430e8514faa2c42de104'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:53.613658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-dd775bfe-ac1f-4581-a25a-ed4a95abb783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f04baf2739c0430e8514faa2c42de104'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:53.628615 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-dd775bfe-ac1f-4581-a25a-ed4a95abb783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f04baf2739c0430e8514faa2c42de104'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:53.645229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ccadcd-7d27-4758-8952-216884c456ea service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:53.656273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ccadcd-7d27-4758-8952-216884c456ea service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:53.661632 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebe8c2-a7a0-4684-893a-08f7bf0a8648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:53.662469 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebe8c2-a7a0-4684-893a-08f7bf0a8648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:53.662759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebe8c2-a7a0-4684-893a-08f7bf0a8648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:53.663191 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebe8c2-a7a0-4684-893a-08f7bf0a8648 admin admin] PATH_INFO: `/v3/users/98783397ee9a4d799a5541f447287da2` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:53.664547 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ebe8c2-a7a0-4684-893a-08f7bf0a8648 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98783397ee9a4d799a5541f447287da2)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:53.666570 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ebe8c2-a7a0-4684-893a-08f7bf0a8648 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:53.743359 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-dd775bfe-ac1f-4581-a25a-ed4a95abb783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f04baf2739c0430e8514faa2c42de104'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:53.762142 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-dd775bfe-ac1f-4581-a25a-ed4a95abb783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f04baf2739c0430e8514faa2c42de104'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:53.762142 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 887/1775] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:06:53 2022] DELETE /identity/v3/users/f04baf2739c0430e8514faa2c42de104 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:53.762142 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268b7f6-0be9-4c2e-98c7-17a9a71f7cbc None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:53.762935 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268b7f6-0be9-4c2e-98c7-17a9a71f7cbc None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:53.763073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268b7f6-0be9-4c2e-98c7-17a9a71f7cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:53.795867 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a268b7f6-0be9-4c2e-98c7-17a9a71f7cbc None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:53.798612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a268b7f6-0be9-4c2e-98c7-17a9a71f7cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:53.888401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 888/1776] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:06:53.899236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268b7f6-0be9-4c2e-98c7-17a9a71f7cbc None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:53.901286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a268b7f6-0be9-4c2e-98c7-17a9a71f7cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:53.936135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:53.990638 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-07ebe8c2-a7a0-4684-893a-08f7bf0a8648 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98783397ee9a4d799a5541f447287da2'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:54.005225 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268b7f6-0be9-4c2e-98c7-17a9a71f7cbc None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:54.008618 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-07ebe8c2-a7a0-4684-893a-08f7bf0a8648 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98783397ee9a4d799a5541f447287da2'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:54.010206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861967d-4fce-4c97-b703-710c9cf805d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:54.011380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861967d-4fce-4c97-b703-710c9cf805d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:54.012153 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861967d-4fce-4c97-b703-710c9cf805d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:54.012438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861967d-4fce-4c97-b703-710c9cf805d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:54.020057 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-07ebe8c2-a7a0-4684-893a-08f7bf0a8648 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98783397ee9a4d799a5541f447287da2'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:54.036602 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861967d-4fce-4c97-b703-710c9cf805d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:54.044131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861967d-4fce-4c97-b703-710c9cf805d4 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:54.049985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 889/1777] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:54.131885 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-07ebe8c2-a7a0-4684-893a-08f7bf0a8648 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98783397ee9a4d799a5541f447287da2'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:54.132611 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-07ebe8c2-a7a0-4684-893a-08f7bf0a8648 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98783397ee9a4d799a5541f447287da2'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:54.139818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 889/1778] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:06:53 2022] DELETE /identity/v3/users/98783397ee9a4d799a5541f447287da2 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:06:54.145088 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029126c-7605-49f5-93a4-3b4324a58d9a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:54.145456 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029126c-7605-49f5-93a4-3b4324a58d9a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:54.145818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029126c-7605-49f5-93a4-3b4324a58d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:54.167969 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9029126c-7605-49f5-93a4-3b4324a58d9a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:54.169333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9029126c-7605-49f5-93a4-3b4324a58d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:54.237125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 890/1779] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:06:54.247610 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebe8c2-a7a0-4684-893a-08f7bf0a8648 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:54.249176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-07ebe8c2-a7a0-4684-893a-08f7bf0a8648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:54.337109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebe8c2-a7a0-4684-893a-08f7bf0a8648 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:54.342466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd993854-ef65-431e-9317-cd7c319c0d0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:54.343100 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd993854-ef65-431e-9317-cd7c319c0d0a service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:54.343393 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd993854-ef65-431e-9317-cd7c319c0d0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:54.343671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd993854-ef65-431e-9317-cd7c319c0d0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:54.355259 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd993854-ef65-431e-9317-cd7c319c0d0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:54.356764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd993854-ef65-431e-9317-cd7c319c0d0a service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:54.367502 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 890/1780] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:54.889303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029126c-7605-49f5-93a4-3b4324a58d9a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:54.890888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9029126c-7605-49f5-93a4-3b4324a58d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:54.906030 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029126c-7605-49f5-93a4-3b4324a58d9a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:54.912685 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b81a07-4d04-4b72-ab5d-da45b4ca2eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:54.913341 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b81a07-4d04-4b72-ab5d-da45b4ca2eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:54.913628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b81a07-4d04-4b72-ab5d-da45b4ca2eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:54.913947 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b81a07-4d04-4b72-ab5d-da45b4ca2eeb admin admin] PATH_INFO: `/v3/projects/1b87db87166c4c58ac35e2b66ed7bc7e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:54.915100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b81a07-4d04-4b72-ab5d-da45b4ca2eeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b87db87166c4c58ac35e2b66ed7bc7e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:54.917067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b81a07-4d04-4b72-ab5d-da45b4ca2eeb admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:55.003277 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd993854-ef65-431e-9317-cd7c319c0d0a service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:55.004840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cd993854-ef65-431e-9317-cd7c319c0d0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:55.021617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd993854-ef65-431e-9317-cd7c319c0d0a service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:55.028928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94684d7a-9c3f-4c34-9fa1-74c425806c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:55.029518 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94684d7a-9c3f-4c34-9fa1-74c425806c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:55.029822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94684d7a-9c3f-4c34-9fa1-74c425806c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:55.030147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94684d7a-9c3f-4c34-9fa1-74c425806c6e admin admin] PATH_INFO: `/v3/projects/c9b0af5514c94e178bc37e1ea5053060` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:55.031273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94684d7a-9c3f-4c34-9fa1-74c425806c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9b0af5514c94e178bc37e1ea5053060)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:55.033961 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94684d7a-9c3f-4c34-9fa1-74c425806c6e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:55.135439 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c9b81a07-4d04-4b72-ab5d-da45b4ca2eeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b87db87166c4c58ac35e2b66ed7bc7e', 'request_id': 'req-c9b81a07-4d04-4b72-ab5d-da45b4ca2eeb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.171598 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-94684d7a-9c3f-4c34-9fa1-74c425806c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9b0af5514c94e178bc37e1ea5053060', 'request_id': 'req-94684d7a-9c3f-4c34-9fa1-74c425806c6e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.281533 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c9b81a07-4d04-4b72-ab5d-da45b4ca2eeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b87db87166c4c58ac35e2b66ed7bc7e', 'request_id': 'req-c9b81a07-4d04-4b72-ab5d-da45b4ca2eeb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.288233 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c9b81a07-4d04-4b72-ab5d-da45b4ca2eeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b87db87166c4c58ac35e2b66ed7bc7e', 'request_id': 'req-c9b81a07-4d04-4b72-ab5d-da45b4ca2eeb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.300258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c9b81a07-4d04-4b72-ab5d-da45b4ca2eeb admin admin] The token cache is being invalidate because project 1b87db87166c4c58ac35e2b66ed7bc7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:06:55.300557 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c9b81a07-4d04-4b72-ab5d-da45b4ca2eeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.301234 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c9b81a07-4d04-4b72-ab5d-da45b4ca2eeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.302827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 891/1781] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:06:54 2022] DELETE /identity/v3/projects/1b87db87166c4c58ac35e2b66ed7bc7e => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:55.306673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b81a07-4d04-4b72-ab5d-da45b4ca2eeb admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:55.308097 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c9b81a07-4d04-4b72-ab5d-da45b4ca2eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:55.355712 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-94684d7a-9c3f-4c34-9fa1-74c425806c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9b0af5514c94e178bc37e1ea5053060', 'request_id': 'req-94684d7a-9c3f-4c34-9fa1-74c425806c6e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.359545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:55.364063 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-94684d7a-9c3f-4c34-9fa1-74c425806c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9b0af5514c94e178bc37e1ea5053060', 'request_id': 'req-94684d7a-9c3f-4c34-9fa1-74c425806c6e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.372167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:55.379719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-94684d7a-9c3f-4c34-9fa1-74c425806c6e admin admin] The token cache is being invalidate because project c9b0af5514c94e178bc37e1ea5053060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:06:55.380129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-94684d7a-9c3f-4c34-9fa1-74c425806c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.380299 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b81a07-4d04-4b72-ab5d-da45b4ca2eeb admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:55.381027 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-94684d7a-9c3f-4c34-9fa1-74c425806c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.382860 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 891/1782] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:06:55 2022] DELETE /identity/v3/projects/c9b0af5514c94e178bc37e1ea5053060 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:55.387259 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94684d7a-9c3f-4c34-9fa1-74c425806c6e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:55.388674 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-94684d7a-9c3f-4c34-9fa1-74c425806c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:55.450368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bf356-b8e6-45c0-beb5-b24546adf009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:55.451121 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bf356-b8e6-45c0-beb5-b24546adf009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:55.451516 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bf356-b8e6-45c0-beb5-b24546adf009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:55.451872 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bf356-b8e6-45c0-beb5-b24546adf009 admin admin] PATH_INFO: `/v3/users/0d9abac08970410992485358655cde87` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:55.453225 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bf356-b8e6-45c0-beb5-b24546adf009 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d9abac08970410992485358655cde87)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:55.455876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bf356-b8e6-45c0-beb5-b24546adf009 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:55.491541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94684d7a-9c3f-4c34-9fa1-74c425806c6e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:55.497229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62ddf1-3a78-4223-9b61-f1c792e73c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:55.497850 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62ddf1-3a78-4223-9b61-f1c792e73c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:55.498188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62ddf1-3a78-4223-9b61-f1c792e73c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:55.498468 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62ddf1-3a78-4223-9b61-f1c792e73c96 admin admin] PATH_INFO: `/v3/users/8cd9e39ac2e3488f8fabb78e8807399e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:55.500052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62ddf1-3a78-4223-9b61-f1c792e73c96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cd9e39ac2e3488f8fabb78e8807399e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:55.502759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62ddf1-3a78-4223-9b61-f1c792e73c96 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:55.702878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fd8bf356-b8e6-45c0-beb5-b24546adf009 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d9abac08970410992485358655cde87'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.722019 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fd8bf356-b8e6-45c0-beb5-b24546adf009 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d9abac08970410992485358655cde87'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.742236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fd8bf356-b8e6-45c0-beb5-b24546adf009 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d9abac08970410992485358655cde87'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.749475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2d62ddf1-3a78-4223-9b61-f1c792e73c96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cd9e39ac2e3488f8fabb78e8807399e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.772131 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2d62ddf1-3a78-4223-9b61-f1c792e73c96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cd9e39ac2e3488f8fabb78e8807399e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.780097 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2d62ddf1-3a78-4223-9b61-f1c792e73c96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cd9e39ac2e3488f8fabb78e8807399e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.851415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fd8bf356-b8e6-45c0-beb5-b24546adf009 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d9abac08970410992485358655cde87'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.852188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fd8bf356-b8e6-45c0-beb5-b24546adf009 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d9abac08970410992485358655cde87'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.864691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 892/1783] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:06:55 2022] DELETE /identity/v3/users/0d9abac08970410992485358655cde87 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:06:55.964042 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2d62ddf1-3a78-4223-9b61-f1c792e73c96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cd9e39ac2e3488f8fabb78e8807399e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.965433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2d62ddf1-3a78-4223-9b61-f1c792e73c96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cd9e39ac2e3488f8fabb78e8807399e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:55.979622 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 892/1784] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:06:55 2022] DELETE /identity/v3/users/8cd9e39ac2e3488f8fabb78e8807399e => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:56.414383 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bf356-b8e6-45c0-beb5-b24546adf009 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:56.414866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fd8bf356-b8e6-45c0-beb5-b24546adf009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:56.499148 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bf356-b8e6-45c0-beb5-b24546adf009 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:56.517725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:56.562323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d9785-8091-4501-80f3-b6c7665a217f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:56.563016 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d9785-8091-4501-80f3-b6c7665a217f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:56.563335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d9785-8091-4501-80f3-b6c7665a217f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:56.563717 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d9785-8091-4501-80f3-b6c7665a217f admin admin] PATH_INFO: `/v3/projects/aa46bc3c534f4d4bbe7db6c3b315b5f9` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:56.565733 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d9785-8091-4501-80f3-b6c7665a217f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa46bc3c534f4d4bbe7db6c3b315b5f9)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:56.567793 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d9785-8091-4501-80f3-b6c7665a217f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:56.603254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62ddf1-3a78-4223-9b61-f1c792e73c96 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:56.604741 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2d62ddf1-3a78-4223-9b61-f1c792e73c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:56.620612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62ddf1-3a78-4223-9b61-f1c792e73c96 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:56.626962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613469e-223f-43ac-b868-b0aaeb0a46e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:56.627812 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613469e-223f-43ac-b868-b0aaeb0a46e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:56.627931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613469e-223f-43ac-b868-b0aaeb0a46e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:56.628189 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613469e-223f-43ac-b868-b0aaeb0a46e3 admin admin] PATH_INFO: `/v3/projects/696f03908243454d88e1beb80a580e83` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:56.629373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613469e-223f-43ac-b868-b0aaeb0a46e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=696f03908243454d88e1beb80a580e83)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:56.633656 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613469e-223f-43ac-b868-b0aaeb0a46e3 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:56.695573 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-df7d9785-8091-4501-80f3-b6c7665a217f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa46bc3c534f4d4bbe7db6c3b315b5f9', 'request_id': 'req-df7d9785-8091-4501-80f3-b6c7665a217f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:56.760399 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1613469e-223f-43ac-b868-b0aaeb0a46e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '696f03908243454d88e1beb80a580e83', 'request_id': 'req-1613469e-223f-43ac-b868-b0aaeb0a46e3'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:56.789750 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-df7d9785-8091-4501-80f3-b6c7665a217f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa46bc3c534f4d4bbe7db6c3b315b5f9', 'request_id': 'req-df7d9785-8091-4501-80f3-b6c7665a217f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:56.801266 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-df7d9785-8091-4501-80f3-b6c7665a217f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa46bc3c534f4d4bbe7db6c3b315b5f9', 'request_id': 'req-df7d9785-8091-4501-80f3-b6c7665a217f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:56.817267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-df7d9785-8091-4501-80f3-b6c7665a217f admin admin] The token cache is being invalidate because project aa46bc3c534f4d4bbe7db6c3b315b5f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:06:56.823907 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-df7d9785-8091-4501-80f3-b6c7665a217f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:56.823907 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-df7d9785-8091-4501-80f3-b6c7665a217f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:56.823907 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 893/1785] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:06:56 2022] DELETE /identity/v3/projects/aa46bc3c534f4d4bbe7db6c3b315b5f9 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:56.827136 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b9fc4-bb7d-4e2c-94e8-ef45d2a7b5ae None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:56.827956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b9fc4-bb7d-4e2c-94e8-ef45d2a7b5ae None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:56.828619 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b9fc4-bb7d-4e2c-94e8-ef45d2a7b5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:56.853314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-173b9fc4-bb7d-4e2c-94e8-ef45d2a7b5ae None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:56.856110 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-173b9fc4-bb7d-4e2c-94e8-ef45d2a7b5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:56.868862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1613469e-223f-43ac-b868-b0aaeb0a46e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '696f03908243454d88e1beb80a580e83', 'request_id': 'req-1613469e-223f-43ac-b868-b0aaeb0a46e3'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:56.879557 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1613469e-223f-43ac-b868-b0aaeb0a46e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '696f03908243454d88e1beb80a580e83', 'request_id': 'req-1613469e-223f-43ac-b868-b0aaeb0a46e3'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:56.891688 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:56.898485 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1613469e-223f-43ac-b868-b0aaeb0a46e3 admin admin] The token cache is being invalidate because project 696f03908243454d88e1beb80a580e83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:06:56.899117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1613469e-223f-43ac-b868-b0aaeb0a46e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:56.900549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1613469e-223f-43ac-b868-b0aaeb0a46e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:06:56.903792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 893/1786] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:06:56 2022] DELETE /identity/v3/projects/696f03908243454d88e1beb80a580e83 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:06:56.911864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028cd64-1335-4e22-a559-5bfec8042294 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:56.912447 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028cd64-1335-4e22-a559-5bfec8042294 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:56.912945 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028cd64-1335-4e22-a559-5bfec8042294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:56.942737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 894/1787] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:56.943419 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4028cd64-1335-4e22-a559-5bfec8042294 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:56.944995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4028cd64-1335-4e22-a559-5bfec8042294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:56.955716 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8d6bc-6949-47b2-b8da-e21f0a4f9684 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:56.955888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8d6bc-6949-47b2-b8da-e21f0a4f9684 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:56.956145 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8d6bc-6949-47b2-b8da-e21f0a4f9684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:56.979416 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 894/1788] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:56.981391 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990e87c-0d51-4337-9a8e-bd9cbe21ed09 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:56.981735 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990e87c-0d51-4337-9a8e-bd9cbe21ed09 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:56.986001 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990e87c-0d51-4337-9a8e-bd9cbe21ed09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:56.986141 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-12c8d6bc-6949-47b2-b8da-e21f0a4f9684 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:56.986141 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-12c8d6bc-6949-47b2-b8da-e21f0a4f9684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:56.997766 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 895/1789] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:57.000118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8d6bc-6949-47b2-b8da-e21f0a4f9684 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:57.010895 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e990e87c-0d51-4337-9a8e-bd9cbe21ed09 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:57.012532 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e990e87c-0d51-4337-9a8e-bd9cbe21ed09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:57.014981 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8d6bc-6949-47b2-b8da-e21f0a4f9684 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:57.034123 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7f6fd-6521-49d0-9750-2e3249912aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:57.034123 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7f6fd-6521-49d0-9750-2e3249912aaa admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:57.034503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 895/1790] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:06:57.034587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7f6fd-6521-49d0-9750-2e3249912aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:57.034587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7f6fd-6521-49d0-9750-2e3249912aaa admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:57.034587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7f6fd-6521-49d0-9750-2e3249912aaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:06:57.034587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7f6fd-6521-49d0-9750-2e3249912aaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:57.034587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7f6fd-6521-49d0-9750-2e3249912aaa admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:57.037630 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990e87c-0d51-4337-9a8e-bd9cbe21ed09 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:57.047367 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990e87c-0d51-4337-9a8e-bd9cbe21ed09 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:57.052690 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 896/1791] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 12:06:57.056144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7f6fd-6521-49d0-9750-2e3249912aaa admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:57.057588 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c8c98-a137-4423-8134-caf62fd5fea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:57.058261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c8c98-a137-4423-8134-caf62fd5fea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:57.058849 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c8c98-a137-4423-8134-caf62fd5fea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:57.058849 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c8c98-a137-4423-8134-caf62fd5fea9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:57.059507 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c8c98-a137-4423-8134-caf62fd5fea9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:06:57.059819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c8c98-a137-4423-8134-caf62fd5fea9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:57.061628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c8c98-a137-4423-8134-caf62fd5fea9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:57.070525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7f6fd-6521-49d0-9750-2e3249912aaa admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:57.076318 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 896/1792] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 12:06:57.076852 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4150e-32e1-47ae-ba8e-133aeeb76327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:57.077495 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4150e-32e1-47ae-ba8e-133aeeb76327 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:57.077766 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4150e-32e1-47ae-ba8e-133aeeb76327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:57.078072 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4150e-32e1-47ae-ba8e-133aeeb76327 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:57.078761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4150e-32e1-47ae-ba8e-133aeeb76327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:57.080733 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c8c98-a137-4423-8134-caf62fd5fea9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:57.081157 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4150e-32e1-47ae-ba8e-133aeeb76327 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:57.093511 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c8c98-a137-4423-8134-caf62fd5fea9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:57.100969 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262c794-59d0-4a68-8551-07b1c4b0e9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:57.102036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262c794-59d0-4a68-8551-07b1c4b0e9f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:57.102541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262c794-59d0-4a68-8551-07b1c4b0e9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:57.103204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262c794-59d0-4a68-8551-07b1c4b0e9f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:57.104284 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262c794-59d0-4a68-8551-07b1c4b0e9f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:57.107301 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262c794-59d0-4a68-8551-07b1c4b0e9f1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:57.197588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 897/1793] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:06:57 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:06:57.198844 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4150e-32e1-47ae-ba8e-133aeeb76327 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:57.209584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:57.226755 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 897/1794] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:06:57 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:06:57.232606 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262c794-59d0-4a68-8551-07b1c4b0e9f1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:57.286952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4150e-32e1-47ae-ba8e-133aeeb76327 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:57.323275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262c794-59d0-4a68-8551-07b1c4b0e9f1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:57.331368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db34b5f-5632-441f-b8dd-545a98c2a52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:57.332100 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db34b5f-5632-441f-b8dd-545a98c2a52b admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:57.332455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db34b5f-5632-441f-b8dd-545a98c2a52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:57.332809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db34b5f-5632-441f-b8dd-545a98c2a52b admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:57.333509 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db34b5f-5632-441f-b8dd-545a98c2a52b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:57.335961 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db34b5f-5632-441f-b8dd-545a98c2a52b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:57.363486 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6909987-82ef-48cb-a00a-1de4d1699d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:57.364109 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6909987-82ef-48cb-a00a-1de4d1699d77 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:57.364453 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6909987-82ef-48cb-a00a-1de4d1699d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:57.364732 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6909987-82ef-48cb-a00a-1de4d1699d77 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:57.365414 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6909987-82ef-48cb-a00a-1de4d1699d77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:57.367443 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6909987-82ef-48cb-a00a-1de4d1699d77 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:57.470902 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 898/1795] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:06:57 2022] POST /identity/v3/users => generated 292 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:06:57.475009 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db34b5f-5632-441f-b8dd-545a98c2a52b admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:57.484843 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db34b5f-5632-441f-b8dd-545a98c2a52b admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:57.489538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0fc19-c5d5-401e-b4e0-85343023e919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:57.490183 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0fc19-c5d5-401e-b4e0-85343023e919 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:57.490466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0fc19-c5d5-401e-b4e0-85343023e919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:57.490744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0fc19-c5d5-401e-b4e0-85343023e919 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:57.491396 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0fc19-c5d5-401e-b4e0-85343023e919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:57.492985 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0fc19-c5d5-401e-b4e0-85343023e919 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:57.504299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 899/1796] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:06:57.508216 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0fc19-c5d5-401e-b4e0-85343023e919 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:57.510066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 898/1797] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:06:57 2022] POST /identity/v3/users => generated 283 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:06:57.519615 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6909987-82ef-48cb-a00a-1de4d1699d77 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:57.519802 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0fc19-c5d5-401e-b4e0-85343023e919 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:57.523742 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d48f2-6099-4cb0-8613-9de8f833a434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:57.524344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d48f2-6099-4cb0-8613-9de8f833a434 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:57.524881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d48f2-6099-4cb0-8613-9de8f833a434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:57.525184 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d48f2-6099-4cb0-8613-9de8f833a434 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:57.525798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d48f2-6099-4cb0-8613-9de8f833a434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:57.526665 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6909987-82ef-48cb-a00a-1de4d1699d77 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:57.527413 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d48f2-6099-4cb0-8613-9de8f833a434 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:57.531801 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9ecab-8432-49f9-aec4-fa426329592b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:57.532561 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9ecab-8432-49f9-aec4-fa426329592b admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:57.532874 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9ecab-8432-49f9-aec4-fa426329592b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:57.533153 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9ecab-8432-49f9-aec4-fa426329592b admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:57.533804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9ecab-8432-49f9-aec4-fa426329592b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:57.535439 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9ecab-8432-49f9-aec4-fa426329592b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:57.538365 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 900/1798] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:06:57.543023 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d48f2-6099-4cb0-8613-9de8f833a434 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:57.555436 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d48f2-6099-4cb0-8613-9de8f833a434 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:57.555648 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 899/1799] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:06:57.560633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9ecab-8432-49f9-aec4-fa426329592b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:57.564489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da3c9e-7345-4df9-a268-f9fc11d7e887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:57.565532 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da3c9e-7345-4df9-a268-f9fc11d7e887 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:57.565645 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da3c9e-7345-4df9-a268-f9fc11d7e887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:57.565841 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da3c9e-7345-4df9-a268-f9fc11d7e887 admin admin] PATH_INFO: `/v3/projects/e17f48350cc04dce8fdc40875bef63da/users/2823c6c0fff84677bb2fe7331c13b0ba/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:57.571733 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9ecab-8432-49f9-aec4-fa426329592b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:57.577055 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa78d0f-065c-43cd-af88-5a74268b7e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:57.578118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa78d0f-065c-43cd-af88-5a74268b7e40 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:57.578118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa78d0f-065c-43cd-af88-5a74268b7e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:57.578381 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa78d0f-065c-43cd-af88-5a74268b7e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:57.578898 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa78d0f-065c-43cd-af88-5a74268b7e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:57.581196 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa78d0f-065c-43cd-af88-5a74268b7e40 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:57.585265 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da3c9e-7345-4df9-a268-f9fc11d7e887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e17f48350cc04dce8fdc40875bef63da, user_id=2823c6c0fff84677bb2fe7331c13b0ba, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:57.587968 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da3c9e-7345-4df9-a268-f9fc11d7e887 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:57.595291 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 900/1800] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:06:57.600167 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa78d0f-065c-43cd-af88-5a74268b7e40 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:57.615071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa78d0f-065c-43cd-af88-5a74268b7e40 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:57.620118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51b508-cb9f-4eb2-a217-18f0396dbf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:57.620792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51b508-cb9f-4eb2-a217-18f0396dbf75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:57.621079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51b508-cb9f-4eb2-a217-18f0396dbf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:57.621362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51b508-cb9f-4eb2-a217-18f0396dbf75 admin admin] PATH_INFO: `/v3/projects/d5e04a1349f348039613b324b53de95d/users/49a29a22517142afb2665623390c7ea8/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:57.637920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d51b508-cb9f-4eb2-a217-18f0396dbf75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e04a1349f348039613b324b53de95d, user_id=49a29a22517142afb2665623390c7ea8, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:57.640614 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d51b508-cb9f-4eb2-a217-18f0396dbf75 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:57.707211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 901/1801] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:06:57 2022] PUT /identity/v3/projects/e17f48350cc04dce8fdc40875bef63da/users/2823c6c0fff84677bb2fe7331c13b0ba/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:06:57.718368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eeb760-db12-4731-bd36-dad13e1a1631 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:57.718368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eeb760-db12-4731-bd36-dad13e1a1631 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:57.718368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eeb760-db12-4731-bd36-dad13e1a1631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:57.762484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-82eeb760-db12-4731-bd36-dad13e1a1631 None None] MFA Rules not processed for user `2823c6c0fff84677bb2fe7331c13b0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:57.763849 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-82eeb760-db12-4731-bd36-dad13e1a1631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:57.776939 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 901/1802] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:06:57 2022] PUT /identity/v3/projects/d5e04a1349f348039613b324b53de95d/users/49a29a22517142afb2665623390c7ea8/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:06:57.787551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0eafb2-3cdd-4126-9712-9fb6c6efdfcf None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:57.788049 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0eafb2-3cdd-4126-9712-9fb6c6efdfcf None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:57.788536 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0eafb2-3cdd-4126-9712-9fb6c6efdfcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:57.832822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9a0eafb2-3cdd-4126-9712-9fb6c6efdfcf None None] MFA Rules not processed for user `49a29a22517142afb2665623390c7ea8`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:57.835630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9a0eafb2-3cdd-4126-9712-9fb6c6efdfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:57.844364 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:57.874280 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 902/1803] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:06:57.879815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eeb760-db12-4731-bd36-dad13e1a1631 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:57.910615 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:57.944370 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 902/1804] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:06:57.951191 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0eafb2-3cdd-4126-9712-9fb6c6efdfcf None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:57.957737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eeb760-db12-4731-bd36-dad13e1a1631 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:58.037398 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0eafb2-3cdd-4126-9712-9fb6c6efdfcf None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:58.041779 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ea003-1a52-45e0-869e-23927e8ea7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:58.042590 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ea003-1a52-45e0-869e-23927e8ea7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:58.043071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ea003-1a52-45e0-869e-23927e8ea7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:58.043789 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ea003-1a52-45e0-869e-23927e8ea7c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:58.045188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ea003-1a52-45e0-869e-23927e8ea7c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:58.046816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f288d-74f0-49d5-92e2-5dc2b3533861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:58.047662 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f288d-74f0-49d5-92e2-5dc2b3533861 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:58.049733 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f288d-74f0-49d5-92e2-5dc2b3533861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:58.049951 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ea003-1a52-45e0-869e-23927e8ea7c2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:58.050144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f288d-74f0-49d5-92e2-5dc2b3533861 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:58.050799 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f288d-74f0-49d5-92e2-5dc2b3533861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:58.055138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f288d-74f0-49d5-92e2-5dc2b3533861 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:58.174956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 903/1805] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:06:57 2022] POST /identity/v3/projects => generated 383 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:06:58.180405 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ea003-1a52-45e0-869e-23927e8ea7c2 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:58.204159 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 903/1806] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:06:57 2022] POST /identity/v3/projects => generated 363 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:06:58.208180 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f288d-74f0-49d5-92e2-5dc2b3533861 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:58.301483 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ea003-1a52-45e0-869e-23927e8ea7c2 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:58.324194 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f288d-74f0-49d5-92e2-5dc2b3533861 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:58.330059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:58.331857 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732cf1d7-c322-48a5-9b71-148edd22c539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:58.332501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cf1d7-c322-48a5-9b71-148edd22c539 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:58.332791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cf1d7-c322-48a5-9b71-148edd22c539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:58.333563 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cf1d7-c322-48a5-9b71-148edd22c539 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:58.334230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732cf1d7-c322-48a5-9b71-148edd22c539 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:58.336395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732cf1d7-c322-48a5-9b71-148edd22c539 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:58.383893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d249da-7d74-465b-9fc7-d7d005954bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:58.384531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d249da-7d74-465b-9fc7-d7d005954bcc admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:58.385519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d249da-7d74-465b-9fc7-d7d005954bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:58.385519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d249da-7d74-465b-9fc7-d7d005954bcc admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:58.385788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d249da-7d74-465b-9fc7-d7d005954bcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:58.387791 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d249da-7d74-465b-9fc7-d7d005954bcc admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:58.439703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 904/1807] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:06:58 2022] POST /identity/v3/users => generated 288 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:06:58.443567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732cf1d7-c322-48a5-9b71-148edd22c539 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:58.458432 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732cf1d7-c322-48a5-9b71-148edd22c539 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:58.465912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b40e8-6c27-4030-ac46-9cad81e1b202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:58.465912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b40e8-6c27-4030-ac46-9cad81e1b202 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:58.465912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b40e8-6c27-4030-ac46-9cad81e1b202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:58.465912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b40e8-6c27-4030-ac46-9cad81e1b202 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:58.466842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b40e8-6c27-4030-ac46-9cad81e1b202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:58.469016 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b40e8-6c27-4030-ac46-9cad81e1b202 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:58.481475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 905/1808] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:06:58.485617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b40e8-6c27-4030-ac46-9cad81e1b202 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:58.496260 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b40e8-6c27-4030-ac46-9cad81e1b202 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:58.501717 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc92e8-b47d-4805-9efe-f01cebada4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:58.502675 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc92e8-b47d-4805-9efe-f01cebada4ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:58.502981 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc92e8-b47d-4805-9efe-f01cebada4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:58.503270 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc92e8-b47d-4805-9efe-f01cebada4ab admin admin] PATH_INFO: `/v3/projects/4588505e826e4f3e86f251e447a9f6c9/users/241c82f676fb40f3aab7129b03cded2f/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:58.520218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc92e8-b47d-4805-9efe-f01cebada4ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=4588505e826e4f3e86f251e447a9f6c9, user_id=241c82f676fb40f3aab7129b03cded2f, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:58.523112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc92e8-b47d-4805-9efe-f01cebada4ab admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:58.555845 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 904/1809] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:06:58 2022] POST /identity/v3/users => generated 298 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:06:58.557140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d249da-7d74-465b-9fc7-d7d005954bcc admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:58.568619 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d249da-7d74-465b-9fc7-d7d005954bcc admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:58.575213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330a8a1-fd52-4e82-afe7-da78887ac3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:58.575840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330a8a1-fd52-4e82-afe7-da78887ac3e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:58.576144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330a8a1-fd52-4e82-afe7-da78887ac3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:58.576412 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330a8a1-fd52-4e82-afe7-da78887ac3e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:58.577058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330a8a1-fd52-4e82-afe7-da78887ac3e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:58.578713 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330a8a1-fd52-4e82-afe7-da78887ac3e1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:58.589537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 905/1810] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:06:58.593470 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330a8a1-fd52-4e82-afe7-da78887ac3e1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:58.603681 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330a8a1-fd52-4e82-afe7-da78887ac3e1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:58.608366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd900c-283b-4b6a-b49c-4a2882bbf74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:58.609026 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd900c-283b-4b6a-b49c-4a2882bbf74e admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:58.609308 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd900c-283b-4b6a-b49c-4a2882bbf74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:58.609584 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd900c-283b-4b6a-b49c-4a2882bbf74e admin admin] PATH_INFO: `/v3/projects/937d67b321fe4f838e4867c44f636114/users/b318a112d63d4e36a44e1d1c50a3d398/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:58.627167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd900c-283b-4b6a-b49c-4a2882bbf74e admin admin] RBAC: Authorizing `identity:create_grant(project_id=937d67b321fe4f838e4867c44f636114, user_id=b318a112d63d4e36a44e1d1c50a3d398, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:58.631470 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd900c-283b-4b6a-b49c-4a2882bbf74e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:58.637031 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 906/1811] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:06:58 2022] PUT /identity/v3/projects/4588505e826e4f3e86f251e447a9f6c9/users/241c82f676fb40f3aab7129b03cded2f/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:06:58.641515 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc92e8-b47d-4805-9efe-f01cebada4ab admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:58.677182 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:58.718793 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc92e8-b47d-4805-9efe-f01cebada4ab admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:58.723526 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6353f-5855-4896-bb4a-d6affac135ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:58.724135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6353f-5855-4896-bb4a-d6affac135ac admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:58.724422 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6353f-5855-4896-bb4a-d6affac135ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:58.724701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6353f-5855-4896-bb4a-d6affac135ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:58.725352 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb6353f-5855-4896-bb4a-d6affac135ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:58.726993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb6353f-5855-4896-bb4a-d6affac135ac admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:58.737989 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 907/1812] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:06:58.741664 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6353f-5855-4896-bb4a-d6affac135ac admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:58.745841 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 906/1813] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:06:58 2022] PUT /identity/v3/projects/937d67b321fe4f838e4867c44f636114/users/b318a112d63d4e36a44e1d1c50a3d398/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:06:58.749910 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd900c-283b-4b6a-b49c-4a2882bbf74e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:58.752461 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6353f-5855-4896-bb4a-d6affac135ac admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:58.834977 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd900c-283b-4b6a-b49c-4a2882bbf74e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:58.837698 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7fdc5d-bef5-4ac0-a458-1e06e91128bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:58.838522 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fdc5d-bef5-4ac0-a458-1e06e91128bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:58.838813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fdc5d-bef5-4ac0-a458-1e06e91128bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:58.839102 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fdc5d-bef5-4ac0-a458-1e06e91128bd admin admin] PATH_INFO: `/v3/projects/4588505e826e4f3e86f251e447a9f6c9/users/241c82f676fb40f3aab7129b03cded2f/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:58.840259 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92ca67-ab3c-440e-b9c6-04cdc17bfbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:58.840881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92ca67-ab3c-440e-b9c6-04cdc17bfbc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:58.841167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92ca67-ab3c-440e-b9c6-04cdc17bfbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:58.841256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7fdc5d-bef5-4ac0-a458-1e06e91128bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4588505e826e4f3e86f251e447a9f6c9, user_id=241c82f676fb40f3aab7129b03cded2f, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:58.841445 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92ca67-ab3c-440e-b9c6-04cdc17bfbc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:58.842131 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92ca67-ab3c-440e-b9c6-04cdc17bfbc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:58.843710 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92ca67-ab3c-440e-b9c6-04cdc17bfbc9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:58.844294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7fdc5d-bef5-4ac0-a458-1e06e91128bd admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:58.854136 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 907/1814] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:06:58.855108 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 908/1815] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:06:58 2022] PUT /identity/v3/projects/4588505e826e4f3e86f251e447a9f6c9/users/241c82f676fb40f3aab7129b03cded2f/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:06:58.858543 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92ca67-ab3c-440e-b9c6-04cdc17bfbc9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:58.869639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92ca67-ab3c-440e-b9c6-04cdc17bfbc9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:58.872992 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174f174-5461-4420-a5af-e8e3a362a090 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:58.873284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174f174-5461-4420-a5af-e8e3a362a090 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:58.873706 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174f174-5461-4420-a5af-e8e3a362a090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:58.908962 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-7174f174-5461-4420-a5af-e8e3a362a090 None None] MFA Rules not processed for user `241c82f676fb40f3aab7129b03cded2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:58.910471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7174f174-5461-4420-a5af-e8e3a362a090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:58.970455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffeb8ab-b644-4054-827a-222f5e5f65cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:58.971173 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffeb8ab-b644-4054-827a-222f5e5f65cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:58.971533 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffeb8ab-b644-4054-827a-222f5e5f65cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:58.971738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffeb8ab-b644-4054-827a-222f5e5f65cc admin admin] PATH_INFO: `/v3/projects/937d67b321fe4f838e4867c44f636114/users/b318a112d63d4e36a44e1d1c50a3d398/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:58.974265 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffeb8ab-b644-4054-827a-222f5e5f65cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=937d67b321fe4f838e4867c44f636114, user_id=b318a112d63d4e36a44e1d1c50a3d398, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:58.977940 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffeb8ab-b644-4054-827a-222f5e5f65cc admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:58.991998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 908/1816] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:06:58 2022] PUT /identity/v3/projects/937d67b321fe4f838e4867c44f636114/users/b318a112d63d4e36a44e1d1c50a3d398/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:06:59.002813 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbddac5-39a2-4902-8b58-9f38610f08c8 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:59.003088 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbddac5-39a2-4902-8b58-9f38610f08c8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:59.004085 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbddac5-39a2-4902-8b58-9f38610f08c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:59.020994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 909/1817] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:06:59.028575 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:59.038616 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174f174-5461-4420-a5af-e8e3a362a090 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:59.039398 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-cfbddac5-39a2-4902-8b58-9f38610f08c8 None None] MFA Rules not processed for user `b318a112d63d4e36a44e1d1c50a3d398`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:06:59.040257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7174f174-5461-4420-a5af-e8e3a362a090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:59.040603 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cfbddac5-39a2-4902-8b58-9f38610f08c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:59.108958 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:06:59.125171 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174f174-5461-4420-a5af-e8e3a362a090 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:59.128255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 909/1818] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:06:59 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:06:59.140897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbddac5-39a2-4902-8b58-9f38610f08c8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:59.142230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cfbddac5-39a2-4902-8b58-9f38610f08c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:06:59.180540 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ac123-6335-4f9e-a759-cd2ee1322264 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:59.181243 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ac123-6335-4f9e-a759-cd2ee1322264 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:59.181705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ac123-6335-4f9e-a759-cd2ee1322264 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:59.182032 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ac123-6335-4f9e-a759-cd2ee1322264 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:59.208640 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbddac5-39a2-4902-8b58-9f38610f08c8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:59.212401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501ef28-d67b-459c-b135-3f29c09ebec5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:59.213022 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501ef28-d67b-459c-b135-3f29c09ebec5 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:59.213313 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501ef28-d67b-459c-b135-3f29c09ebec5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:59.213592 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501ef28-d67b-459c-b135-3f29c09ebec5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:59.224589 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501ef28-d67b-459c-b135-3f29c09ebec5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:59.227358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501ef28-d67b-459c-b135-3f29c09ebec5 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:59.240594 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 910/1819] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:59 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:06:59.262155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ac123-6335-4f9e-a759-cd2ee1322264 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:59.262155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ac123-6335-4f9e-a759-cd2ee1322264 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:59.270985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 910/1820] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:06:59.424913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501ef28-d67b-459c-b135-3f29c09ebec5 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:59.437350 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501ef28-d67b-459c-b135-3f29c09ebec5 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:59.445582 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d44d5-bcca-4809-b34c-db604247ed2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:59.445582 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d44d5-bcca-4809-b34c-db604247ed2c service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:59.445582 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d44d5-bcca-4809-b34c-db604247ed2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:59.445582 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d44d5-bcca-4809-b34c-db604247ed2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:59.520615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d44d5-bcca-4809-b34c-db604247ed2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:59.522274 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d44d5-bcca-4809-b34c-db604247ed2c service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:59.533153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 911/1821] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:06:59.600988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ac123-6335-4f9e-a759-cd2ee1322264 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:06:59.617846 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ac123-6335-4f9e-a759-cd2ee1322264 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:06:59.638321 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7c03f-aad2-49f5-9bac-2056142a4790 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:06:59.640535 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7c03f-aad2-49f5-9bac-2056142a4790 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:06:59.645595 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7c03f-aad2-49f5-9bac-2056142a4790 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:06:59.645595 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7c03f-aad2-49f5-9bac-2056142a4790 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:06:59.762426 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7c03f-aad2-49f5-9bac-2056142a4790 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:06:59.764334 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7c03f-aad2-49f5-9bac-2056142a4790 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:06:59.786487 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 911/1822] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:06:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:00.633777 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd05fdb-1346-46de-8c07-220951eb51d3 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:00.634437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd05fdb-1346-46de-8c07-220951eb51d3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:00.634437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd05fdb-1346-46de-8c07-220951eb51d3 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:00.636022 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 912/1823] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 14 12:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:07:00.657707 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a1e2b-c289-461f-b839-6037811b68e6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:00.658257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a1e2b-c289-461f-b839-6037811b68e6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:00.658866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a1e2b-c289-461f-b839-6037811b68e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:00.715201 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-298a1e2b-c289-461f-b839-6037811b68e6 None None] MFA Rules not processed for user `38b093bc4c1d4ebcbd2a6bb16b70c029`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:00.715201 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-298a1e2b-c289-461f-b839-6037811b68e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:00.855175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:00.936049 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 912/1824] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 14 12:07:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:01.498711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd05fdb-1346-46de-8c07-220951eb51d3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:01.500780 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7bd05fdb-1346-46de-8c07-220951eb51d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:01.524891 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd05fdb-1346-46de-8c07-220951eb51d3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:01.535484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15dbcd-1192-4588-adab-220bb22c7226 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:01.536171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15dbcd-1192-4588-adab-220bb22c7226 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:01.536642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15dbcd-1192-4588-adab-220bb22c7226 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:01.536992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15dbcd-1192-4588-adab-220bb22c7226 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:01.595979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15dbcd-1192-4588-adab-220bb22c7226 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:01.600620 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15dbcd-1192-4588-adab-220bb22c7226 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:01.611393 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a1e2b-c289-461f-b839-6037811b68e6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:01.612878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-298a1e2b-c289-461f-b839-6037811b68e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:01.633215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 913/1825] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:01.786925 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a1e2b-c289-461f-b839-6037811b68e6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:01.793991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e35a8-f88e-4311-87d3-354de71ff80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:01.799633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e35a8-f88e-4311-87d3-354de71ff80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:01.799633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e35a8-f88e-4311-87d3-354de71ff80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:01.799633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e35a8-f88e-4311-87d3-354de71ff80a admin admin] PATH_INFO: `/v3/users/a6f788f0db6043d1a7be3068408a68a6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:01.799633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e35a8-f88e-4311-87d3-354de71ff80a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f788f0db6043d1a7be3068408a68a6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:01.799633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e35a8-f88e-4311-87d3-354de71ff80a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:02.055581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-db6e35a8-f88e-4311-87d3-354de71ff80a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f788f0db6043d1a7be3068408a68a6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:02.094537 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-db6e35a8-f88e-4311-87d3-354de71ff80a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f788f0db6043d1a7be3068408a68a6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:02.114649 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-db6e35a8-f88e-4311-87d3-354de71ff80a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f788f0db6043d1a7be3068408a68a6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:02.212668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-db6e35a8-f88e-4311-87d3-354de71ff80a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f788f0db6043d1a7be3068408a68a6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:02.212668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-db6e35a8-f88e-4311-87d3-354de71ff80a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f788f0db6043d1a7be3068408a68a6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:02.218068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 913/1826] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:01 2022] DELETE /identity/v3/users/a6f788f0db6043d1a7be3068408a68a6 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:07:02.223505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de9465-19d2-4ebc-b894-7d0c7a9de35a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:02.225244 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de9465-19d2-4ebc-b894-7d0c7a9de35a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:02.226380 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de9465-19d2-4ebc-b894-7d0c7a9de35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:02.268164 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a2de9465-19d2-4ebc-b894-7d0c7a9de35a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:02.269376 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a2de9465-19d2-4ebc-b894-7d0c7a9de35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:02.294635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:02.381382 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 914/1827] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:02.392732 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e35a8-f88e-4311-87d3-354de71ff80a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:02.395127 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-db6e35a8-f88e-4311-87d3-354de71ff80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:02.561942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e35a8-f88e-4311-87d3-354de71ff80a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:02.567924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c583e-da0b-4e14-ac0c-3a5d46f72135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:02.568986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c583e-da0b-4e14-ac0c-3a5d46f72135 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:02.569505 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c583e-da0b-4e14-ac0c-3a5d46f72135 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:02.570047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c583e-da0b-4e14-ac0c-3a5d46f72135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:02.618550 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c583e-da0b-4e14-ac0c-3a5d46f72135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:02.623798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c583e-da0b-4e14-ac0c-3a5d46f72135 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:02.642956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 914/1828] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:03.287047 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de9465-19d2-4ebc-b894-7d0c7a9de35a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:03.288608 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a2de9465-19d2-4ebc-b894-7d0c7a9de35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:03.316337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de9465-19d2-4ebc-b894-7d0c7a9de35a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:03.321899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea499f2c-dc4c-48d8-8205-b0bd180c1c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:03.322756 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea499f2c-dc4c-48d8-8205-b0bd180c1c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:03.323204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea499f2c-dc4c-48d8-8205-b0bd180c1c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:03.323495 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea499f2c-dc4c-48d8-8205-b0bd180c1c48 admin admin] PATH_INFO: `/v3/projects/0c69b707fee641568ba3e1de7d5c4de1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:03.324782 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea499f2c-dc4c-48d8-8205-b0bd180c1c48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c69b707fee641568ba3e1de7d5c4de1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:03.338156 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea499f2c-dc4c-48d8-8205-b0bd180c1c48 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:03.453423 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ea499f2c-dc4c-48d8-8205-b0bd180c1c48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c69b707fee641568ba3e1de7d5c4de1', 'request_id': 'req-ea499f2c-dc4c-48d8-8205-b0bd180c1c48'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:03.526595 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ea499f2c-dc4c-48d8-8205-b0bd180c1c48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c69b707fee641568ba3e1de7d5c4de1', 'request_id': 'req-ea499f2c-dc4c-48d8-8205-b0bd180c1c48'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:03.534268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ea499f2c-dc4c-48d8-8205-b0bd180c1c48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c69b707fee641568ba3e1de7d5c4de1', 'request_id': 'req-ea499f2c-dc4c-48d8-8205-b0bd180c1c48'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:03.556659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ea499f2c-dc4c-48d8-8205-b0bd180c1c48 admin admin] The token cache is being invalidate because project 0c69b707fee641568ba3e1de7d5c4de1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:03.557074 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ea499f2c-dc4c-48d8-8205-b0bd180c1c48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:03.557824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ea499f2c-dc4c-48d8-8205-b0bd180c1c48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:03.563475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 915/1829] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:03 2022] DELETE /identity/v3/projects/0c69b707fee641568ba3e1de7d5c4de1 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:03.568937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c583e-da0b-4e14-ac0c-3a5d46f72135 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:03.573850 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:03.579266 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c583e-da0b-4e14-ac0c-3a5d46f72135 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:03.647407 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a75a6-6e31-4859-be12-ff92b8cea770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:03.648081 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a75a6-6e31-4859-be12-ff92b8cea770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:03.648367 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a75a6-6e31-4859-be12-ff92b8cea770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:03.648650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a75a6-6e31-4859-be12-ff92b8cea770 admin admin] PATH_INFO: `/v3/users/dff34d763447487b9dfb46c0707b738b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:03.650102 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a75a6-6e31-4859-be12-ff92b8cea770 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dff34d763447487b9dfb46c0707b738b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:03.652082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a75a6-6e31-4859-be12-ff92b8cea770 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:03.814334 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e57a75a6-6e31-4859-be12-ff92b8cea770 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dff34d763447487b9dfb46c0707b738b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:03.837905 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e57a75a6-6e31-4859-be12-ff92b8cea770 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dff34d763447487b9dfb46c0707b738b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:03.850365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e57a75a6-6e31-4859-be12-ff92b8cea770 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dff34d763447487b9dfb46c0707b738b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:03.932538 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e57a75a6-6e31-4859-be12-ff92b8cea770 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dff34d763447487b9dfb46c0707b738b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:03.933178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e57a75a6-6e31-4859-be12-ff92b8cea770 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dff34d763447487b9dfb46c0707b738b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:03.939813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 915/1830] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:03 2022] DELETE /identity/v3/users/dff34d763447487b9dfb46c0707b738b => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:04.393639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea499f2c-dc4c-48d8-8205-b0bd180c1c48 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:04.395578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ea499f2c-dc4c-48d8-8205-b0bd180c1c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:04.410949 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea499f2c-dc4c-48d8-8205-b0bd180c1c48 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:04.476085 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e98ec-9744-45ea-bdbe-46e34ba87112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:04.478215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e98ec-9744-45ea-bdbe-46e34ba87112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:04.478669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e98ec-9744-45ea-bdbe-46e34ba87112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:04.479238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e98ec-9744-45ea-bdbe-46e34ba87112 admin admin] PATH_INFO: `/v3/projects/67bc496988864f4286330e1808e42a5f` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:04.481961 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e98ec-9744-45ea-bdbe-46e34ba87112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67bc496988864f4286330e1808e42a5f)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:04.484909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e98ec-9744-45ea-bdbe-46e34ba87112 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:04.600614 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:04.602151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5f1e98ec-9744-45ea-bdbe-46e34ba87112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67bc496988864f4286330e1808e42a5f', 'request_id': 'req-5f1e98ec-9744-45ea-bdbe-46e34ba87112'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:04.669381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5f1e98ec-9744-45ea-bdbe-46e34ba87112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67bc496988864f4286330e1808e42a5f', 'request_id': 'req-5f1e98ec-9744-45ea-bdbe-46e34ba87112'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:04.677634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5f1e98ec-9744-45ea-bdbe-46e34ba87112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67bc496988864f4286330e1808e42a5f', 'request_id': 'req-5f1e98ec-9744-45ea-bdbe-46e34ba87112'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:04.690743 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5f1e98ec-9744-45ea-bdbe-46e34ba87112 admin admin] The token cache is being invalidate because project 67bc496988864f4286330e1808e42a5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:04.691478 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5f1e98ec-9744-45ea-bdbe-46e34ba87112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:04.692554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5f1e98ec-9744-45ea-bdbe-46e34ba87112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:04.698306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 916/1831] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:04 2022] DELETE /identity/v3/projects/67bc496988864f4286330e1808e42a5f => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:04.701370 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731f8dd-59a4-41b0-b025-8b89a7ed5292 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:04.702142 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731f8dd-59a4-41b0-b025-8b89a7ed5292 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:04.702391 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731f8dd-59a4-41b0-b025-8b89a7ed5292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:04.723986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-7731f8dd-59a4-41b0-b025-8b89a7ed5292 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:04.730833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7731f8dd-59a4-41b0-b025-8b89a7ed5292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:04.807672 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 916/1832] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:04.815776 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137686a-d701-449c-a18c-c40ed0277c54 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:04.816064 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137686a-d701-449c-a18c-c40ed0277c54 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:04.816425 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137686a-d701-449c-a18c-c40ed0277c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:04.846140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0137686a-d701-449c-a18c-c40ed0277c54 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:04.847933 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0137686a-d701-449c-a18c-c40ed0277c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:04.858812 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 917/1833] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:04.863700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731f8dd-59a4-41b0-b025-8b89a7ed5292 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:04.880126 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731f8dd-59a4-41b0-b025-8b89a7ed5292 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:04.886094 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af566bd1-4716-4497-bf92-ee1a97264b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:04.886681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af566bd1-4716-4497-bf92-ee1a97264b7e admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:04.886959 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af566bd1-4716-4497-bf92-ee1a97264b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:04.887235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af566bd1-4716-4497-bf92-ee1a97264b7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:04.887892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af566bd1-4716-4497-bf92-ee1a97264b7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:07:04.888198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af566bd1-4716-4497-bf92-ee1a97264b7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:04.889900 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af566bd1-4716-4497-bf92-ee1a97264b7e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:04.903962 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 917/1834] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 12:07:04.909313 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137686a-d701-449c-a18c-c40ed0277c54 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:04.918697 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137686a-d701-449c-a18c-c40ed0277c54 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:04.923367 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2df023-1e9e-4bf3-8725-3da1f414f508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:04.923969 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2df023-1e9e-4bf3-8725-3da1f414f508 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:04.924251 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2df023-1e9e-4bf3-8725-3da1f414f508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:04.924524 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2df023-1e9e-4bf3-8725-3da1f414f508 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:04.925152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2df023-1e9e-4bf3-8725-3da1f414f508 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:04.927133 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2df023-1e9e-4bf3-8725-3da1f414f508 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:05.029102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 918/1835] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:04 2022] POST /identity/v3/projects => generated 369 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:05.033120 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af566bd1-4716-4497-bf92-ee1a97264b7e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:05.044685 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af566bd1-4716-4497-bf92-ee1a97264b7e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:05.049761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:05.106409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb78695-a603-4da4-a339-20a15fdd3751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:05.107033 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb78695-a603-4da4-a339-20a15fdd3751 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:05.107309 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb78695-a603-4da4-a339-20a15fdd3751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:05.107584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb78695-a603-4da4-a339-20a15fdd3751 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:05.108278 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb78695-a603-4da4-a339-20a15fdd3751 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:05.110250 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb78695-a603-4da4-a339-20a15fdd3751 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:05.166674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 918/1836] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:05 2022] POST /identity/v3/users => generated 285 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:05.172487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2df023-1e9e-4bf3-8725-3da1f414f508 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:05.183885 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2df023-1e9e-4bf3-8725-3da1f414f508 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:05.188479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc1b5e-cc1f-4b2b-b1d2-11999deb50ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:05.189080 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc1b5e-cc1f-4b2b-b1d2-11999deb50ee admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:05.189357 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc1b5e-cc1f-4b2b-b1d2-11999deb50ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:05.189625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc1b5e-cc1f-4b2b-b1d2-11999deb50ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:05.190321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc1b5e-cc1f-4b2b-b1d2-11999deb50ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:05.191901 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc1b5e-cc1f-4b2b-b1d2-11999deb50ee admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:05.201928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 919/1837] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:05.207225 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb78695-a603-4da4-a339-20a15fdd3751 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:05.218882 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb78695-a603-4da4-a339-20a15fdd3751 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:05.225966 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15548af4-0519-486c-a841-de489e31be2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:05.226634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15548af4-0519-486c-a841-de489e31be2b admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:05.226918 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15548af4-0519-486c-a841-de489e31be2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:05.227259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15548af4-0519-486c-a841-de489e31be2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:05.227830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15548af4-0519-486c-a841-de489e31be2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:05.229462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15548af4-0519-486c-a841-de489e31be2b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:05.241620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 919/1838] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:05.245956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc1b5e-cc1f-4b2b-b1d2-11999deb50ee admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:05.257328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc1b5e-cc1f-4b2b-b1d2-11999deb50ee admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:05.262436 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f1541-435e-4ae7-8e69-334f6015c837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:05.263109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f1541-435e-4ae7-8e69-334f6015c837 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:05.263395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f1541-435e-4ae7-8e69-334f6015c837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:05.263669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f1541-435e-4ae7-8e69-334f6015c837 admin admin] PATH_INFO: `/v3/projects/cb805a5354ee4a77be1617aad852c5c7/users/4d9d90357e074564a37d0415cc871550/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:05.283657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f1541-435e-4ae7-8e69-334f6015c837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb805a5354ee4a77be1617aad852c5c7, user_id=4d9d90357e074564a37d0415cc871550, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:05.287445 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f1541-435e-4ae7-8e69-334f6015c837 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:05.363196 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 920/1839] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:05 2022] PUT /identity/v3/projects/cb805a5354ee4a77be1617aad852c5c7/users/4d9d90357e074564a37d0415cc871550/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:05.375076 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5a7c3-f2bd-48e9-908e-8da40de38d33 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:05.375583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5a7c3-f2bd-48e9-908e-8da40de38d33 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:05.375962 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5a7c3-f2bd-48e9-908e-8da40de38d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:05.409996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-66b5a7c3-f2bd-48e9-908e-8da40de38d33 None None] MFA Rules not processed for user `4d9d90357e074564a37d0415cc871550`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:05.411380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-66b5a7c3-f2bd-48e9-908e-8da40de38d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:05.510796 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 920/1840] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:05.515367 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f1541-435e-4ae7-8e69-334f6015c837 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:05.578406 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f1541-435e-4ae7-8e69-334f6015c837 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:05.583567 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd0977-4d89-473c-98a9-a5c0fee93a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:05.584189 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd0977-4d89-473c-98a9-a5c0fee93a4f admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:05.584469 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd0977-4d89-473c-98a9-a5c0fee93a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:05.584747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd0977-4d89-473c-98a9-a5c0fee93a4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:05.585394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd0977-4d89-473c-98a9-a5c0fee93a4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:05.587331 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd0977-4d89-473c-98a9-a5c0fee93a4f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:05.686158 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 921/1841] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:05 2022] POST /identity/v3/projects => generated 369 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:07:05.687484 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5a7c3-f2bd-48e9-908e-8da40de38d33 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:05.699364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5a7c3-f2bd-48e9-908e-8da40de38d33 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:05.764669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc206e-d448-472d-80b4-702e64dec215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:05.765353 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc206e-d448-472d-80b4-702e64dec215 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:05.765611 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc206e-d448-472d-80b4-702e64dec215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:05.765920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc206e-d448-472d-80b4-702e64dec215 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:05.767556 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc206e-d448-472d-80b4-702e64dec215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:05.769992 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc206e-d448-472d-80b4-702e64dec215 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:05.876942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 921/1842] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:05 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:05.881451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd0977-4d89-473c-98a9-a5c0fee93a4f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:05.894859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd0977-4d89-473c-98a9-a5c0fee93a4f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:05.899764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59b121-8aac-4826-b196-3fe955094b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:05.900761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59b121-8aac-4826-b196-3fe955094b8a admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:05.901058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59b121-8aac-4826-b196-3fe955094b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:05.901336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59b121-8aac-4826-b196-3fe955094b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:05.905091 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59b121-8aac-4826-b196-3fe955094b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:05.905091 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59b121-8aac-4826-b196-3fe955094b8a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:05.916507 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 922/1843] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:05.920973 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc206e-d448-472d-80b4-702e64dec215 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:05.939442 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc206e-d448-472d-80b4-702e64dec215 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:05.946519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5644e-8913-48a7-9c3a-b57cedf17318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:05.947293 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5644e-8913-48a7-9c3a-b57cedf17318 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:05.947574 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5644e-8913-48a7-9c3a-b57cedf17318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:05.947848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5644e-8913-48a7-9c3a-b57cedf17318 admin admin] PATH_INFO: `/v3/projects/53c2ec63e22e49e7adb8b3c85bd2a63c/users/709611d9382146739d205383ebe2e7ea/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:05.950555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:05.968232 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5644e-8913-48a7-9c3a-b57cedf17318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53c2ec63e22e49e7adb8b3c85bd2a63c, user_id=709611d9382146739d205383ebe2e7ea, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:05.971031 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5644e-8913-48a7-9c3a-b57cedf17318 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:06.048399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 922/1844] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:05 2022] PUT /identity/v3/projects/53c2ec63e22e49e7adb8b3c85bd2a63c/users/709611d9382146739d205383ebe2e7ea/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:07:06.052424 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59b121-8aac-4826-b196-3fe955094b8a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:06.063695 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59b121-8aac-4826-b196-3fe955094b8a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:06.107472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:06.122549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8af98a-5fd0-40dc-a329-9d816427b598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:06.123207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8af98a-5fd0-40dc-a329-9d816427b598 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:06.123493 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8af98a-5fd0-40dc-a329-9d816427b598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:06.123771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8af98a-5fd0-40dc-a329-9d816427b598 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:06.124429 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8af98a-5fd0-40dc-a329-9d816427b598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:06.126782 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8af98a-5fd0-40dc-a329-9d816427b598 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:06.138414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 923/1845] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:07:06.142676 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5644e-8913-48a7-9c3a-b57cedf17318 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:06.154082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5644e-8913-48a7-9c3a-b57cedf17318 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:06.160641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614f400-531c-400f-935c-c001878c821d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:06.160641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614f400-531c-400f-935c-c001878c821d admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:06.160641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614f400-531c-400f-935c-c001878c821d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:06.160980 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614f400-531c-400f-935c-c001878c821d admin admin] PATH_INFO: `/v3/projects/53c2ec63e22e49e7adb8b3c85bd2a63c/users/709611d9382146739d205383ebe2e7ea/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:06.162854 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614f400-531c-400f-935c-c001878c821d admin admin] RBAC: Authorizing `identity:create_grant(project_id=53c2ec63e22e49e7adb8b3c85bd2a63c, user_id=709611d9382146739d205383ebe2e7ea, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:06.165637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614f400-531c-400f-935c-c001878c821d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:06.176804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 923/1846] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:06 2022] PUT /identity/v3/projects/53c2ec63e22e49e7adb8b3c85bd2a63c/users/709611d9382146739d205383ebe2e7ea/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:06.188903 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c48f6b-fa6a-455e-b4d6-de2a0ddab6f4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:06.189198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c48f6b-fa6a-455e-b4d6-de2a0ddab6f4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:06.189485 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c48f6b-fa6a-455e-b4d6-de2a0ddab6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:06.219622 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-72c48f6b-fa6a-455e-b4d6-de2a0ddab6f4 None None] MFA Rules not processed for user `709611d9382146739d205383ebe2e7ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:06.220860 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-72c48f6b-fa6a-455e-b4d6-de2a0ddab6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:06.301420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 924/1847] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:06.315421 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614f400-531c-400f-935c-c001878c821d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:06.317353 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6614f400-531c-400f-935c-c001878c821d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:06.384710 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614f400-531c-400f-935c-c001878c821d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:06.388618 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e5627-0420-47c4-9b13-c90ca86b69b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:06.389240 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5627-0420-47c4-9b13-c90ca86b69b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:06.389525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5627-0420-47c4-9b13-c90ca86b69b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:06.389801 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5627-0420-47c4-9b13-c90ca86b69b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:06.402568 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e5627-0420-47c4-9b13-c90ca86b69b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:06.404405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e5627-0420-47c4-9b13-c90ca86b69b1 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:06.416283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 924/1848] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:07.676889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c48f6b-fa6a-455e-b4d6-de2a0ddab6f4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:07.680299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-72c48f6b-fa6a-455e-b4d6-de2a0ddab6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:07.888641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c48f6b-fa6a-455e-b4d6-de2a0ddab6f4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:07.908058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f02d0-efc5-424b-95bd-0ead315fb810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:07.908954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f02d0-efc5-424b-95bd-0ead315fb810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:07.909246 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f02d0-efc5-424b-95bd-0ead315fb810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:07.909522 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f02d0-efc5-424b-95bd-0ead315fb810 admin admin] PATH_INFO: `/v3/users/49a29a22517142afb2665623390c7ea8` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:07.913362 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f02d0-efc5-424b-95bd-0ead315fb810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49a29a22517142afb2665623390c7ea8)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:07.916286 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f02d0-efc5-424b-95bd-0ead315fb810 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:08.163012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7e7f02d0-efc5-424b-95bd-0ead315fb810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49a29a22517142afb2665623390c7ea8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:08.191289 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7e7f02d0-efc5-424b-95bd-0ead315fb810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49a29a22517142afb2665623390c7ea8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:08.196811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:08.200321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7e7f02d0-efc5-424b-95bd-0ead315fb810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49a29a22517142afb2665623390c7ea8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:08.281547 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7e7f02d0-efc5-424b-95bd-0ead315fb810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49a29a22517142afb2665623390c7ea8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:08.282762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7e7f02d0-efc5-424b-95bd-0ead315fb810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49a29a22517142afb2665623390c7ea8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:08.297295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 925/1849] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:07 2022] DELETE /identity/v3/users/49a29a22517142afb2665623390c7ea8 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:08.303386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4123cac-3d15-49b1-b306-03b482a4cd7a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:08.304594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4123cac-3d15-49b1-b306-03b482a4cd7a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:08.304594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4123cac-3d15-49b1-b306-03b482a4cd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:08.325917 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c4123cac-3d15-49b1-b306-03b482a4cd7a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:08.330393 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c4123cac-3d15-49b1-b306-03b482a4cd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:08.408308 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 925/1850] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:08.422568 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f02d0-efc5-424b-95bd-0ead315fb810 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:08.424227 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7e7f02d0-efc5-424b-95bd-0ead315fb810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:08.535652 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f02d0-efc5-424b-95bd-0ead315fb810 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:08.547584 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc2e57-12f1-4b2b-9f5f-6995a2e69629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:08.548253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc2e57-12f1-4b2b-9f5f-6995a2e69629 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:08.548536 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc2e57-12f1-4b2b-9f5f-6995a2e69629 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:08.548811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc2e57-12f1-4b2b-9f5f-6995a2e69629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:08.562603 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bc2e57-12f1-4b2b-9f5f-6995a2e69629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:08.564177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bc2e57-12f1-4b2b-9f5f-6995a2e69629 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:08.581449 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 926/1851] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:08.948186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4123cac-3d15-49b1-b306-03b482a4cd7a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:08.949807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c4123cac-3d15-49b1-b306-03b482a4cd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:08.963948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4123cac-3d15-49b1-b306-03b482a4cd7a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:08.971567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bbc4d-26f8-415b-bebd-a940486aaf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:08.972722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bbc4d-26f8-415b-bebd-a940486aaf35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:08.973472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bbc4d-26f8-415b-bebd-a940486aaf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:08.973951 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bbc4d-26f8-415b-bebd-a940486aaf35 admin admin] PATH_INFO: `/v3/users/2823c6c0fff84677bb2fe7331c13b0ba` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:08.976061 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bbc4d-26f8-415b-bebd-a940486aaf35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2823c6c0fff84677bb2fe7331c13b0ba)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:08.979928 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bbc4d-26f8-415b-bebd-a940486aaf35 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:09.183193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc2e57-12f1-4b2b-9f5f-6995a2e69629 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:09.193753 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-56bc2e57-12f1-4b2b-9f5f-6995a2e69629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:09.220704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc2e57-12f1-4b2b-9f5f-6995a2e69629 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:09.229050 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cbfe9-931c-4579-a998-bcdf7b688286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:09.229686 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cbfe9-931c-4579-a998-bcdf7b688286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:09.230005 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cbfe9-931c-4579-a998-bcdf7b688286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:09.230479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cbfe9-931c-4579-a998-bcdf7b688286 admin admin] PATH_INFO: `/v3/projects/d5e04a1349f348039613b324b53de95d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:09.231618 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22cbfe9-931c-4579-a998-bcdf7b688286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5e04a1349f348039613b324b53de95d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:09.233927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22cbfe9-931c-4579-a998-bcdf7b688286 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:09.288471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0b4bbc4d-26f8-415b-bebd-a940486aaf35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2823c6c0fff84677bb2fe7331c13b0ba'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:09.313823 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0b4bbc4d-26f8-415b-bebd-a940486aaf35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2823c6c0fff84677bb2fe7331c13b0ba'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:09.316411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:09.325252 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0b4bbc4d-26f8-415b-bebd-a940486aaf35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2823c6c0fff84677bb2fe7331c13b0ba'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:09.408055 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0b4bbc4d-26f8-415b-bebd-a940486aaf35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2823c6c0fff84677bb2fe7331c13b0ba'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:09.409047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0b4bbc4d-26f8-415b-bebd-a940486aaf35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2823c6c0fff84677bb2fe7331c13b0ba'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:09.414761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e22cbfe9-931c-4579-a998-bcdf7b688286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5e04a1349f348039613b324b53de95d', 'request_id': 'req-e22cbfe9-931c-4579-a998-bcdf7b688286'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:09.418166 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 926/1852] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:08 2022] DELETE /identity/v3/users/2823c6c0fff84677bb2fe7331c13b0ba => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:09.424052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9a72a-b60c-42cc-b49a-9e9b81c72ce8 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:09.424450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9a72a-b60c-42cc-b49a-9e9b81c72ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:09.424961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9a72a-b60c-42cc-b49a-9e9b81c72ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:09.456550 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d1f9a72a-b60c-42cc-b49a-9e9b81c72ce8 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:09.459436 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d1f9a72a-b60c-42cc-b49a-9e9b81c72ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:09.521727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e22cbfe9-931c-4579-a998-bcdf7b688286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5e04a1349f348039613b324b53de95d', 'request_id': 'req-e22cbfe9-931c-4579-a998-bcdf7b688286'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:09.543410 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e22cbfe9-931c-4579-a998-bcdf7b688286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5e04a1349f348039613b324b53de95d', 'request_id': 'req-e22cbfe9-931c-4579-a998-bcdf7b688286'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:09.569295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e22cbfe9-931c-4579-a998-bcdf7b688286 admin admin] The token cache is being invalidate because project d5e04a1349f348039613b324b53de95d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:09.573531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e22cbfe9-931c-4579-a998-bcdf7b688286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:09.573531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e22cbfe9-931c-4579-a998-bcdf7b688286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:09.574275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 927/1853] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:09 2022] DELETE /identity/v3/projects/d5e04a1349f348039613b324b53de95d => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:09.578499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cbfe9-931c-4579-a998-bcdf7b688286 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:09.580506 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e22cbfe9-931c-4579-a998-bcdf7b688286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:09.606001 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 927/1854] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:09.622297 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9a72a-b60c-42cc-b49a-9e9b81c72ce8 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:09.624711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d1f9a72a-b60c-42cc-b49a-9e9b81c72ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:09.674565 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cbfe9-931c-4579-a998-bcdf7b688286 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:09.679613 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d7d7e-a470-4e65-812a-0ea4df321dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:09.680270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d7d7e-a470-4e65-812a-0ea4df321dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:09.680551 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d7d7e-a470-4e65-812a-0ea4df321dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:09.680825 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d7d7e-a470-4e65-812a-0ea4df321dc6 admin admin] PATH_INFO: `/v3/users/241c82f676fb40f3aab7129b03cded2f` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:09.681989 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d7d7e-a470-4e65-812a-0ea4df321dc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=241c82f676fb40f3aab7129b03cded2f)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:09.683909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d7d7e-a470-4e65-812a-0ea4df321dc6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:09.700727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:09.702665 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9a72a-b60c-42cc-b49a-9e9b81c72ce8 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:09.706701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bae6c4-73af-4918-a103-361438b85279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:09.710480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bae6c4-73af-4918-a103-361438b85279 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:09.710725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bae6c4-73af-4918-a103-361438b85279 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:09.711009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bae6c4-73af-4918-a103-361438b85279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:09.730202 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bae6c4-73af-4918-a103-361438b85279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:09.730450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bae6c4-73af-4918-a103-361438b85279 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:09.746360 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 928/1855] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:07:09.950784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1e5d7d7e-a470-4e65-812a-0ea4df321dc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '241c82f676fb40f3aab7129b03cded2f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:09.968249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1e5d7d7e-a470-4e65-812a-0ea4df321dc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '241c82f676fb40f3aab7129b03cded2f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:09.977522 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1e5d7d7e-a470-4e65-812a-0ea4df321dc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '241c82f676fb40f3aab7129b03cded2f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:10.080281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1e5d7d7e-a470-4e65-812a-0ea4df321dc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '241c82f676fb40f3aab7129b03cded2f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:10.083515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1e5d7d7e-a470-4e65-812a-0ea4df321dc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '241c82f676fb40f3aab7129b03cded2f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:10.110457 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 928/1856] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:09 2022] DELETE /identity/v3/users/241c82f676fb40f3aab7129b03cded2f => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:07:10.262828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bae6c4-73af-4918-a103-361438b85279 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:10.265698 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-61bae6c4-73af-4918-a103-361438b85279 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:10.321408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bae6c4-73af-4918-a103-361438b85279 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:10.340691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d7d7e-a470-4e65-812a-0ea4df321dc6 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:10.344042 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1e5d7d7e-a470-4e65-812a-0ea4df321dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:10.448733 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc97f9-08f7-4176-8fd4-5b5075220367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:10.451079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc97f9-08f7-4176-8fd4-5b5075220367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:10.452089 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc97f9-08f7-4176-8fd4-5b5075220367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:10.453040 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc97f9-08f7-4176-8fd4-5b5075220367 admin admin] PATH_INFO: `/v3/projects/e17f48350cc04dce8fdc40875bef63da` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:10.455866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc97f9-08f7-4176-8fd4-5b5075220367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e17f48350cc04dce8fdc40875bef63da)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:10.460844 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc97f9-08f7-4176-8fd4-5b5075220367 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:10.515627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d7d7e-a470-4e65-812a-0ea4df321dc6 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:10.522514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e07128-952e-4d0a-8d4c-156ca40feca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:10.523270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e07128-952e-4d0a-8d4c-156ca40feca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:10.523636 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e07128-952e-4d0a-8d4c-156ca40feca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:10.523993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e07128-952e-4d0a-8d4c-156ca40feca0 admin admin] PATH_INFO: `/v3/users/4d9d90357e074564a37d0415cc871550` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:10.525694 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e07128-952e-4d0a-8d4c-156ca40feca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d9d90357e074564a37d0415cc871550)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:10.527945 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e07128-952e-4d0a-8d4c-156ca40feca0 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:10.638987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:10.677408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-febc97f9-08f7-4176-8fd4-5b5075220367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e17f48350cc04dce8fdc40875bef63da', 'request_id': 'req-febc97f9-08f7-4176-8fd4-5b5075220367'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:10.801986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-febc97f9-08f7-4176-8fd4-5b5075220367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e17f48350cc04dce8fdc40875bef63da', 'request_id': 'req-febc97f9-08f7-4176-8fd4-5b5075220367'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:10.806563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e1e07128-952e-4d0a-8d4c-156ca40feca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d9d90357e074564a37d0415cc871550'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:10.810766 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-febc97f9-08f7-4176-8fd4-5b5075220367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e17f48350cc04dce8fdc40875bef63da', 'request_id': 'req-febc97f9-08f7-4176-8fd4-5b5075220367'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:10.834724 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-febc97f9-08f7-4176-8fd4-5b5075220367 admin admin] The token cache is being invalidate because project e17f48350cc04dce8fdc40875bef63da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:10.835314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-febc97f9-08f7-4176-8fd4-5b5075220367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:10.836676 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-febc97f9-08f7-4176-8fd4-5b5075220367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:10.838692 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 929/1857] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:10 2022] DELETE /identity/v3/projects/e17f48350cc04dce8fdc40875bef63da => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:10.840758 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e1e07128-952e-4d0a-8d4c-156ca40feca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d9d90357e074564a37d0415cc871550'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:10.843118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc97f9-08f7-4176-8fd4-5b5075220367 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:10.845746 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-febc97f9-08f7-4176-8fd4-5b5075220367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:10.854416 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e1e07128-952e-4d0a-8d4c-156ca40feca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d9d90357e074564a37d0415cc871550'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:10.926384 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc97f9-08f7-4176-8fd4-5b5075220367 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:10.931174 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7cfbf-7d72-4b24-bda6-f55295b6fb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:10.931813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7cfbf-7d72-4b24-bda6-f55295b6fb1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:10.932101 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7cfbf-7d72-4b24-bda6-f55295b6fb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:10.932383 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7cfbf-7d72-4b24-bda6-f55295b6fb1a admin admin] PATH_INFO: `/v3/users/b318a112d63d4e36a44e1d1c50a3d398` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:10.933539 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7cfbf-7d72-4b24-bda6-f55295b6fb1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b318a112d63d4e36a44e1d1c50a3d398)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:10.935518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7cfbf-7d72-4b24-bda6-f55295b6fb1a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:10.971440 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e1e07128-952e-4d0a-8d4c-156ca40feca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d9d90357e074564a37d0415cc871550'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:10.972238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e1e07128-952e-4d0a-8d4c-156ca40feca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d9d90357e074564a37d0415cc871550'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:10.983257 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 929/1858] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:10 2022] DELETE /identity/v3/users/4d9d90357e074564a37d0415cc871550 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:07:10.984084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e07128-952e-4d0a-8d4c-156ca40feca0 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:10.986005 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e1e07128-952e-4d0a-8d4c-156ca40feca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:11.006727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:11.072285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e07128-952e-4d0a-8d4c-156ca40feca0 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:11.077502 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78816102-193c-410e-ad6b-3e89e9ea2841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:11.078325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78816102-193c-410e-ad6b-3e89e9ea2841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:11.079249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78816102-193c-410e-ad6b-3e89e9ea2841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:11.095345 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78816102-193c-410e-ad6b-3e89e9ea2841 admin admin] PATH_INFO: `/v3/projects/4588505e826e4f3e86f251e447a9f6c9` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:11.095345 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78816102-193c-410e-ad6b-3e89e9ea2841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4588505e826e4f3e86f251e447a9f6c9)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:11.095345 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78816102-193c-410e-ad6b-3e89e9ea2841 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:11.237572 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-40c7cfbf-7d72-4b24-bda6-f55295b6fb1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b318a112d63d4e36a44e1d1c50a3d398'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:11.239048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-78816102-193c-410e-ad6b-3e89e9ea2841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4588505e826e4f3e86f251e447a9f6c9', 'request_id': 'req-78816102-193c-410e-ad6b-3e89e9ea2841'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:11.255016 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-40c7cfbf-7d72-4b24-bda6-f55295b6fb1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b318a112d63d4e36a44e1d1c50a3d398'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:11.263069 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-40c7cfbf-7d72-4b24-bda6-f55295b6fb1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b318a112d63d4e36a44e1d1c50a3d398'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:11.338974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-78816102-193c-410e-ad6b-3e89e9ea2841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4588505e826e4f3e86f251e447a9f6c9', 'request_id': 'req-78816102-193c-410e-ad6b-3e89e9ea2841'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:11.349327 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-78816102-193c-410e-ad6b-3e89e9ea2841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4588505e826e4f3e86f251e447a9f6c9', 'request_id': 'req-78816102-193c-410e-ad6b-3e89e9ea2841'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:11.364280 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-78816102-193c-410e-ad6b-3e89e9ea2841 admin admin] The token cache is being invalidate because project 4588505e826e4f3e86f251e447a9f6c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:11.364651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-78816102-193c-410e-ad6b-3e89e9ea2841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:11.365367 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-78816102-193c-410e-ad6b-3e89e9ea2841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:11.367150 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 930/1859] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:10 2022] DELETE /identity/v3/projects/4588505e826e4f3e86f251e447a9f6c9 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:11.368452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11c022-9008-454d-985d-66f7d5d78428 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:11.368738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11c022-9008-454d-985d-66f7d5d78428 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:11.369018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11c022-9008-454d-985d-66f7d5d78428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:11.397430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ea11c022-9008-454d-985d-66f7d5d78428 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:11.398704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ea11c022-9008-454d-985d-66f7d5d78428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:11.451200 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-40c7cfbf-7d72-4b24-bda6-f55295b6fb1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b318a112d63d4e36a44e1d1c50a3d398'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:11.454526 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-40c7cfbf-7d72-4b24-bda6-f55295b6fb1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b318a112d63d4e36a44e1d1c50a3d398'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:11.469907 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 930/1860] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:10 2022] DELETE /identity/v3/users/b318a112d63d4e36a44e1d1c50a3d398 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:11.469907 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c8be7-8d43-42f8-8817-683144fba853 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:11.469907 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c8be7-8d43-42f8-8817-683144fba853 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:11.469907 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c8be7-8d43-42f8-8817-683144fba853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:11.477483 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 931/1861] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:11.487750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11c022-9008-454d-985d-66f7d5d78428 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:11.497206 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ea11c022-9008-454d-985d-66f7d5d78428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:11.522902 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a22c8be7-8d43-42f8-8817-683144fba853 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:11.528461 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a22c8be7-8d43-42f8-8817-683144fba853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:11.595046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11c022-9008-454d-985d-66f7d5d78428 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:11.659794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 931/1862] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:11.667629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70621ea-4dea-4369-88e2-7c74d03bd989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:11.667841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5bacc-2878-4225-bb25-b1cf36542853 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:11.668047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5bacc-2878-4225-bb25-b1cf36542853 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:11.668369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5bacc-2878-4225-bb25-b1cf36542853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:11.668944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70621ea-4dea-4369-88e2-7c74d03bd989 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:11.669235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70621ea-4dea-4369-88e2-7c74d03bd989 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:11.669513 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70621ea-4dea-4369-88e2-7c74d03bd989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:11.680964 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70621ea-4dea-4369-88e2-7c74d03bd989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:11.682176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70621ea-4dea-4369-88e2-7c74d03bd989 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:11.689841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c8f5bacc-2878-4225-bb25-b1cf36542853 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:11.691232 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c8f5bacc-2878-4225-bb25-b1cf36542853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:11.693343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 932/1863] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:11.705098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 932/1864] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:11.709282 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70621ea-4dea-4369-88e2-7c74d03bd989 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:11.720642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70621ea-4dea-4369-88e2-7c74d03bd989 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:11.725880 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6679e-a76a-4921-ac10-ef6fd0dfb4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:11.726461 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6679e-a76a-4921-ac10-ef6fd0dfb4e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:11.726845 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6679e-a76a-4921-ac10-ef6fd0dfb4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:11.727284 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6679e-a76a-4921-ac10-ef6fd0dfb4e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:11.727697 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6679e-a76a-4921-ac10-ef6fd0dfb4e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:07:11.728557 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6679e-a76a-4921-ac10-ef6fd0dfb4e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:11.729898 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6679e-a76a-4921-ac10-ef6fd0dfb4e1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:11.749233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 933/1865] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:07:11.752033 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5bacc-2878-4225-bb25-b1cf36542853 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:11.763690 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5bacc-2878-4225-bb25-b1cf36542853 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:11.768939 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fb441-28a1-4ed0-b57e-05faa4b3cd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:11.769702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fb441-28a1-4ed0-b57e-05faa4b3cd1b admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:11.770128 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fb441-28a1-4ed0-b57e-05faa4b3cd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:11.770495 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fb441-28a1-4ed0-b57e-05faa4b3cd1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:11.771267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228fb441-28a1-4ed0-b57e-05faa4b3cd1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:11.773327 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228fb441-28a1-4ed0-b57e-05faa4b3cd1b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:11.798642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:11.920542 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 933/1866] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:11 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:11.927140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6679e-a76a-4921-ac10-ef6fd0dfb4e1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:11.948672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6679e-a76a-4921-ac10-ef6fd0dfb4e1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:12.020596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1675126-cc35-4e9c-9240-314c2512a1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:12.021249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1675126-cc35-4e9c-9240-314c2512a1f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:12.021561 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1675126-cc35-4e9c-9240-314c2512a1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:12.021885 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1675126-cc35-4e9c-9240-314c2512a1f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:12.022657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1675126-cc35-4e9c-9240-314c2512a1f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:12.025086 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1675126-cc35-4e9c-9240-314c2512a1f2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:12.106567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fb441-28a1-4ed0-b57e-05faa4b3cd1b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:12.108395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-228fb441-28a1-4ed0-b57e-05faa4b3cd1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:12.135078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fb441-28a1-4ed0-b57e-05faa4b3cd1b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:12.148033 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d840c28-6a4c-43c2-9e22-46b6d37281ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:12.148437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 934/1867] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:11 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:12.149059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d840c28-6a4c-43c2-9e22-46b6d37281ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:12.149903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d840c28-6a4c-43c2-9e22-46b6d37281ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:12.150471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d840c28-6a4c-43c2-9e22-46b6d37281ed admin admin] PATH_INFO: `/v3/projects/937d67b321fe4f838e4867c44f636114` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:12.153102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1675126-cc35-4e9c-9240-314c2512a1f2 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:12.155405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d840c28-6a4c-43c2-9e22-46b6d37281ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=937d67b321fe4f838e4867c44f636114)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:12.160368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d840c28-6a4c-43c2-9e22-46b6d37281ed admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:12.178267 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1675126-cc35-4e9c-9240-314c2512a1f2 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:12.190699 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff67d56-ec01-4be0-8419-f9980e3267d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:12.192154 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff67d56-ec01-4be0-8419-f9980e3267d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:12.193357 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff67d56-ec01-4be0-8419-f9980e3267d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:12.206600 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff67d56-ec01-4be0-8419-f9980e3267d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:12.207952 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff67d56-ec01-4be0-8419-f9980e3267d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:12.222483 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff67d56-ec01-4be0-8419-f9980e3267d5 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:12.256152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 935/1868] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:07:12.260960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff67d56-ec01-4be0-8419-f9980e3267d5 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:12.287204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff67d56-ec01-4be0-8419-f9980e3267d5 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:12.301435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67788f-ff31-4463-aece-e0f138bc23b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:12.303071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67788f-ff31-4463-aece-e0f138bc23b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:12.303702 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67788f-ff31-4463-aece-e0f138bc23b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:12.304206 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67788f-ff31-4463-aece-e0f138bc23b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:12.305495 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c67788f-ff31-4463-aece-e0f138bc23b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:12.308834 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c67788f-ff31-4463-aece-e0f138bc23b5 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:12.331424 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 936/1869] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:12.338480 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67788f-ff31-4463-aece-e0f138bc23b5 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:12.363414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67788f-ff31-4463-aece-e0f138bc23b5 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:12.380875 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3d840c28-6a4c-43c2-9e22-46b6d37281ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '937d67b321fe4f838e4867c44f636114', 'request_id': 'req-3d840c28-6a4c-43c2-9e22-46b6d37281ed'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:12.446591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb0e5c-f7a1-44c9-9d31-c1438a8d4dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:12.447257 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb0e5c-f7a1-44c9-9d31-c1438a8d4dd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:12.447537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb0e5c-f7a1-44c9-9d31-c1438a8d4dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:12.447810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb0e5c-f7a1-44c9-9d31-c1438a8d4dd4 admin admin] PATH_INFO: `/v3/projects/1c942433245d423199cc295d416bbbcc/users/822b5ba1d3864908852d711222c6a72d/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:12.465255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb0e5c-f7a1-44c9-9d31-c1438a8d4dd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c942433245d423199cc295d416bbbcc, user_id=822b5ba1d3864908852d711222c6a72d, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:12.468003 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb0e5c-f7a1-44c9-9d31-c1438a8d4dd4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:12.487266 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3d840c28-6a4c-43c2-9e22-46b6d37281ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '937d67b321fe4f838e4867c44f636114', 'request_id': 'req-3d840c28-6a4c-43c2-9e22-46b6d37281ed'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:12.496380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3d840c28-6a4c-43c2-9e22-46b6d37281ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '937d67b321fe4f838e4867c44f636114', 'request_id': 'req-3d840c28-6a4c-43c2-9e22-46b6d37281ed'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:12.508627 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3d840c28-6a4c-43c2-9e22-46b6d37281ed admin admin] The token cache is being invalidate because project 937d67b321fe4f838e4867c44f636114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:12.508904 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3d840c28-6a4c-43c2-9e22-46b6d37281ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:12.509597 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3d840c28-6a4c-43c2-9e22-46b6d37281ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:12.511158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 934/1870] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:12 2022] DELETE /identity/v3/projects/937d67b321fe4f838e4867c44f636114 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:12.515689 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:12.523294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42110085-7349-4fae-90f2-a9b8d0b4c3de None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:12.523294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42110085-7349-4fae-90f2-a9b8d0b4c3de None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:12.523294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42110085-7349-4fae-90f2-a9b8d0b4c3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:12.550165 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-42110085-7349-4fae-90f2-a9b8d0b4c3de None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:12.551628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-42110085-7349-4fae-90f2-a9b8d0b4c3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:12.561432 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 935/1871] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:12.564781 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42110085-7349-4fae-90f2-a9b8d0b4c3de None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:12.565759 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-42110085-7349-4fae-90f2-a9b8d0b4c3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:12.589174 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42110085-7349-4fae-90f2-a9b8d0b4c3de None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:12.596312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 937/1872] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:12 2022] PUT /identity/v3/projects/1c942433245d423199cc295d416bbbcc/users/822b5ba1d3864908852d711222c6a72d/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:07:12.597443 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be30ab3-83a3-429b-a016-6957583d7e17 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:12.597757 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be30ab3-83a3-429b-a016-6957583d7e17 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:12.598095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be30ab3-83a3-429b-a016-6957583d7e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:12.621549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6be30ab3-83a3-429b-a016-6957583d7e17 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:12.622746 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6be30ab3-83a3-429b-a016-6957583d7e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:12.668511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b243fc-5b7c-45d0-b6a1-c06f1886c5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:12.669380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b243fc-5b7c-45d0-b6a1-c06f1886c5e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:12.669664 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b243fc-5b7c-45d0-b6a1-c06f1886c5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:12.669992 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b243fc-5b7c-45d0-b6a1-c06f1886c5e0 admin admin] PATH_INFO: `/v3/projects/cb805a5354ee4a77be1617aad852c5c7` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:12.671162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b243fc-5b7c-45d0-b6a1-c06f1886c5e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb805a5354ee4a77be1617aad852c5c7)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:12.673809 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b243fc-5b7c-45d0-b6a1-c06f1886c5e0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:12.688544 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 938/1873] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:12.692616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278e658-5e70-405e-b42e-353ba6007e99 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:12.693077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278e658-5e70-405e-b42e-353ba6007e99 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:12.693521 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278e658-5e70-405e-b42e-353ba6007e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:12.727615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4278e658-5e70-405e-b42e-353ba6007e99 None None] MFA Rules not processed for user `822b5ba1d3864908852d711222c6a72d`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:12.728851 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4278e658-5e70-405e-b42e-353ba6007e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:12.800505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 939/1874] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:12.801452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278e658-5e70-405e-b42e-353ba6007e99 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:12.817264 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278e658-5e70-405e-b42e-353ba6007e99 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:12.831401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-15b243fc-5b7c-45d0-b6a1-c06f1886c5e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb805a5354ee4a77be1617aad852c5c7', 'request_id': 'req-15b243fc-5b7c-45d0-b6a1-c06f1886c5e0'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:12.892812 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c5659-8577-46c5-a075-594c20eba3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:12.893448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c5659-8577-46c5-a075-594c20eba3fa admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:12.893730 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c5659-8577-46c5-a075-594c20eba3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:12.894027 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c5659-8577-46c5-a075-594c20eba3fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:12.894816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c5659-8577-46c5-a075-594c20eba3fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:07:12.895131 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c5659-8577-46c5-a075-594c20eba3fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:12.896811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c5659-8577-46c5-a075-594c20eba3fa admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:12.910837 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 940/1875] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:07:12.911749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c5659-8577-46c5-a075-594c20eba3fa admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:12.916184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-15b243fc-5b7c-45d0-b6a1-c06f1886c5e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb805a5354ee4a77be1617aad852c5c7', 'request_id': 'req-15b243fc-5b7c-45d0-b6a1-c06f1886c5e0'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:12.922721 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e98c5659-8577-46c5-a075-594c20eba3fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:12.923820 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-15b243fc-5b7c-45d0-b6a1-c06f1886c5e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb805a5354ee4a77be1617aad852c5c7', 'request_id': 'req-15b243fc-5b7c-45d0-b6a1-c06f1886c5e0'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:12.942026 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c5659-8577-46c5-a075-594c20eba3fa admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:12.942768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-15b243fc-5b7c-45d0-b6a1-c06f1886c5e0 admin admin] The token cache is being invalidate because project cb805a5354ee4a77be1617aad852c5c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:12.943066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-15b243fc-5b7c-45d0-b6a1-c06f1886c5e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:12.944051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-15b243fc-5b7c-45d0-b6a1-c06f1886c5e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:12.945595 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 936/1876] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:12 2022] DELETE /identity/v3/projects/cb805a5354ee4a77be1617aad852c5c7 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:12.946633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b243fc-5b7c-45d0-b6a1-c06f1886c5e0 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:12.950161 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-15b243fc-5b7c-45d0-b6a1-c06f1886c5e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:12.950608 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6d2ff-c1fd-435d-8857-c206d9467a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:12.951444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6d2ff-c1fd-435d-8857-c206d9467a83 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:12.951784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6d2ff-c1fd-435d-8857-c206d9467a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:12.952062 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6d2ff-c1fd-435d-8857-c206d9467a83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:12.952727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6d2ff-c1fd-435d-8857-c206d9467a83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:12.954946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6d2ff-c1fd-435d-8857-c206d9467a83 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:12.973921 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b243fc-5b7c-45d0-b6a1-c06f1886c5e0 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:12.979820 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0db18-7d62-4f66-96da-4283c0b5670a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:12.980449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0db18-7d62-4f66-96da-4283c0b5670a admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:12.980737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0db18-7d62-4f66-96da-4283c0b5670a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:12.981017 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0db18-7d62-4f66-96da-4283c0b5670a admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:12.981667 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0db18-7d62-4f66-96da-4283c0b5670a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:12.983756 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0db18-7d62-4f66-96da-4283c0b5670a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:13.082102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 941/1877] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:12 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:13.083149 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6d2ff-c1fd-435d-8857-c206d9467a83 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:13.085261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b0d6d2ff-c1fd-435d-8857-c206d9467a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:13.093815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 937/1878] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:12 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:13.093815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0db18-7d62-4f66-96da-4283c0b5670a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:13.097020 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9cd0db18-7d62-4f66-96da-4283c0b5670a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:13.177476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6d2ff-c1fd-435d-8857-c206d9467a83 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:13.197798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0db18-7d62-4f66-96da-4283c0b5670a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:13.203697 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcec94-eee7-41ab-a820-77319693abc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:13.204674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcec94-eee7-41ab-a820-77319693abc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:13.205086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcec94-eee7-41ab-a820-77319693abc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:13.205409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcec94-eee7-41ab-a820-77319693abc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:13.206132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edcec94-eee7-41ab-a820-77319693abc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:13.208113 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edcec94-eee7-41ab-a820-77319693abc0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:13.239274 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b6be5-8f35-4a3a-8e34-a8c29af56b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:13.239968 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b6be5-8f35-4a3a-8e34-a8c29af56b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:13.240251 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b6be5-8f35-4a3a-8e34-a8c29af56b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:13.240529 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b6be5-8f35-4a3a-8e34-a8c29af56b9c admin admin] PATH_INFO: `/v3/users/709611d9382146739d205383ebe2e7ea` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:13.241724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b6be5-8f35-4a3a-8e34-a8c29af56b9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=709611d9382146739d205383ebe2e7ea)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:13.243854 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b6be5-8f35-4a3a-8e34-a8c29af56b9c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:13.296679 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 938/1879] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:13 2022] POST /identity/v3/users => generated 300 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:13.296679 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcec94-eee7-41ab-a820-77319693abc0 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:13.306057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcec94-eee7-41ab-a820-77319693abc0 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:13.310922 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22aabd-9039-47ed-8a0f-d92a684c292b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:13.311639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22aabd-9039-47ed-8a0f-d92a684c292b admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:13.312031 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22aabd-9039-47ed-8a0f-d92a684c292b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:13.312405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22aabd-9039-47ed-8a0f-d92a684c292b admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:13.313643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22aabd-9039-47ed-8a0f-d92a684c292b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:13.315738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22aabd-9039-47ed-8a0f-d92a684c292b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:13.400824 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 939/1880] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:13 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:13.402850 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22aabd-9039-47ed-8a0f-d92a684c292b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:13.414488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22aabd-9039-47ed-8a0f-d92a684c292b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:13.420200 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2831e-e3b5-4894-a586-005e7fc99f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:13.425545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2831e-e3b5-4894-a586-005e7fc99f51 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:13.425545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2831e-e3b5-4894-a586-005e7fc99f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:13.425545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2831e-e3b5-4894-a586-005e7fc99f51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:13.425545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2831e-e3b5-4894-a586-005e7fc99f51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:13.425545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2831e-e3b5-4894-a586-005e7fc99f51 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:13.438621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 940/1881] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:13.439615 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2831e-e3b5-4894-a586-005e7fc99f51 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:13.452457 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2831e-e3b5-4894-a586-005e7fc99f51 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:13.459669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017f6b2-274d-481f-b93c-274a52d0b8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:13.461481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017f6b2-274d-481f-b93c-274a52d0b8fe admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:13.461944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017f6b2-274d-481f-b93c-274a52d0b8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:13.462371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017f6b2-274d-481f-b93c-274a52d0b8fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:13.463211 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017f6b2-274d-481f-b93c-274a52d0b8fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:13.465009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017f6b2-274d-481f-b93c-274a52d0b8fe admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:13.476361 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 941/1882] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:13.477689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017f6b2-274d-481f-b93c-274a52d0b8fe admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:13.500720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017f6b2-274d-481f-b93c-274a52d0b8fe admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:13.505663 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7b4b6be5-8f35-4a3a-8e34-a8c29af56b9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '709611d9382146739d205383ebe2e7ea'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:13.507357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183098f-7c56-4c94-94b2-e5c4b4340484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:13.508272 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183098f-7c56-4c94-94b2-e5c4b4340484 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:13.508798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183098f-7c56-4c94-94b2-e5c4b4340484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:13.509060 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183098f-7c56-4c94-94b2-e5c4b4340484 admin admin] PATH_INFO: `/v3/projects/344d17db4b3a4f329e9fd1fd7eb6a2b1/users/e74fc6c7f0b84e0aa152aee992e81378/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:13.522205 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7b4b6be5-8f35-4a3a-8e34-a8c29af56b9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '709611d9382146739d205383ebe2e7ea'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:13.528070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183098f-7c56-4c94-94b2-e5c4b4340484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=344d17db4b3a4f329e9fd1fd7eb6a2b1, user_id=e74fc6c7f0b84e0aa152aee992e81378, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:13.530635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7b4b6be5-8f35-4a3a-8e34-a8c29af56b9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '709611d9382146739d205383ebe2e7ea'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:13.532236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183098f-7c56-4c94-94b2-e5c4b4340484 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:13.629132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7b4b6be5-8f35-4a3a-8e34-a8c29af56b9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '709611d9382146739d205383ebe2e7ea'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:13.629833 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7b4b6be5-8f35-4a3a-8e34-a8c29af56b9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '709611d9382146739d205383ebe2e7ea'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:13.636498 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 942/1883] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:13 2022] DELETE /identity/v3/users/709611d9382146739d205383ebe2e7ea => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:13.637353 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 942/1884] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:13 2022] PUT /identity/v3/projects/344d17db4b3a4f329e9fd1fd7eb6a2b1/users/e74fc6c7f0b84e0aa152aee992e81378/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:13.640426 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b6be5-8f35-4a3a-8e34-a8c29af56b9c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:13.641770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7b4b6be5-8f35-4a3a-8e34-a8c29af56b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:13.641993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183098f-7c56-4c94-94b2-e5c4b4340484 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:13.643758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9183098f-7c56-4c94-94b2-e5c4b4340484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:13.711965 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b6be5-8f35-4a3a-8e34-a8c29af56b9c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:13.717381 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183098f-7c56-4c94-94b2-e5c4b4340484 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:13.717581 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6cdb92-ce9e-4193-9114-d6778fbf4d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:13.718217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cdb92-ce9e-4193-9114-d6778fbf4d2f admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:13.718538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cdb92-ce9e-4193-9114-d6778fbf4d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:13.718772 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cdb92-ce9e-4193-9114-d6778fbf4d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:13.719463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6cdb92-ce9e-4193-9114-d6778fbf4d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:13.721079 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6cdb92-ce9e-4193-9114-d6778fbf4d2f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:13.722778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c05aa-e077-42ab-a8b5-1fb8c3db30a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:13.725123 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c05aa-e077-42ab-a8b5-1fb8c3db30a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:13.725123 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c05aa-e077-42ab-a8b5-1fb8c3db30a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:13.725123 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c05aa-e077-42ab-a8b5-1fb8c3db30a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:13.725123 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c05aa-e077-42ab-a8b5-1fb8c3db30a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:13.726670 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c05aa-e077-42ab-a8b5-1fb8c3db30a0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:13.733085 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 943/1885] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:13.737424 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6cdb92-ce9e-4193-9114-d6778fbf4d2f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:13.743204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 943/1886] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:07:13.745313 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c05aa-e077-42ab-a8b5-1fb8c3db30a0 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:13.748778 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6cdb92-ce9e-4193-9114-d6778fbf4d2f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:13.754188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12fd40-a798-46d4-925d-a3698ed480ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:13.754857 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12fd40-a798-46d4-925d-a3698ed480ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:13.755137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12fd40-a798-46d4-925d-a3698ed480ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:13.755411 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12fd40-a798-46d4-925d-a3698ed480ca admin admin] PATH_INFO: `/v3/projects/528295fa52bf4f5fa8f0bfd56db158d7/users/ba970bd70bd441d798f3048aa9520fd1/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:13.756473 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c05aa-e077-42ab-a8b5-1fb8c3db30a0 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:13.762537 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb62e72-89a6-4ebf-842b-d4c65731c65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:13.763235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb62e72-89a6-4ebf-842b-d4c65731c65b admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:13.763518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb62e72-89a6-4ebf-842b-d4c65731c65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:13.763795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb62e72-89a6-4ebf-842b-d4c65731c65b admin admin] PATH_INFO: `/v3/projects/344d17db4b3a4f329e9fd1fd7eb6a2b1/users/e74fc6c7f0b84e0aa152aee992e81378/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:13.767216 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb62e72-89a6-4ebf-842b-d4c65731c65b admin admin] RBAC: Authorizing `identity:create_grant(project_id=344d17db4b3a4f329e9fd1fd7eb6a2b1, user_id=e74fc6c7f0b84e0aa152aee992e81378, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:13.770084 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb62e72-89a6-4ebf-842b-d4c65731c65b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:13.782865 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12fd40-a798-46d4-925d-a3698ed480ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=528295fa52bf4f5fa8f0bfd56db158d7, user_id=ba970bd70bd441d798f3048aa9520fd1, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:13.783010 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 944/1887] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:13 2022] PUT /identity/v3/projects/344d17db4b3a4f329e9fd1fd7eb6a2b1/users/e74fc6c7f0b84e0aa152aee992e81378/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:13.786843 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12fd40-a798-46d4-925d-a3698ed480ca admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:13.800298 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559ca3b-ffe1-416b-bf64-24bbee08b53c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:13.800298 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559ca3b-ffe1-416b-bf64-24bbee08b53c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:13.800298 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559ca3b-ffe1-416b-bf64-24bbee08b53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:13.832081 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-6559ca3b-ffe1-416b-bf64-24bbee08b53c None None] MFA Rules not processed for user `e74fc6c7f0b84e0aa152aee992e81378`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:13.833506 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6559ca3b-ffe1-416b-bf64-24bbee08b53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:13.865900 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 944/1888] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:13 2022] PUT /identity/v3/projects/528295fa52bf4f5fa8f0bfd56db158d7/users/ba970bd70bd441d798f3048aa9520fd1/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:07:13.866421 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:13.875034 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b333e-4ba0-4cdc-8a77-3fdd8125e209 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:13.876444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b333e-4ba0-4cdc-8a77-3fdd8125e209 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:13.877332 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b333e-4ba0-4cdc-8a77-3fdd8125e209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:13.911544 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d74b333e-4ba0-4cdc-8a77-3fdd8125e209 None None] MFA Rules not processed for user `ba970bd70bd441d798f3048aa9520fd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:13.912820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d74b333e-4ba0-4cdc-8a77-3fdd8125e209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:13.932323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 945/1889] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:13.944594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559ca3b-ffe1-416b-bf64-24bbee08b53c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:13.950126 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6559ca3b-ffe1-416b-bf64-24bbee08b53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:14.013895 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 945/1890] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:14.024620 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b333e-4ba0-4cdc-8a77-3fdd8125e209 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:14.036237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559ca3b-ffe1-416b-bf64-24bbee08b53c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:14.110337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b333e-4ba0-4cdc-8a77-3fdd8125e209 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:14.116148 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0015a-f09e-45df-93a0-98800cece2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:14.116964 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0015a-f09e-45df-93a0-98800cece2e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:14.117884 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0015a-f09e-45df-93a0-98800cece2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:14.118172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0015a-f09e-45df-93a0-98800cece2e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:14.118559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0246273-4db6-49c1-80d6-20db707698d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:14.118844 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0015a-f09e-45df-93a0-98800cece2e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:14.119932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0246273-4db6-49c1-80d6-20db707698d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:14.120684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0246273-4db6-49c1-80d6-20db707698d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:14.120684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0246273-4db6-49c1-80d6-20db707698d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:14.122410 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0015a-f09e-45df-93a0-98800cece2e5 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:14.202642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0246273-4db6-49c1-80d6-20db707698d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:14.204147 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0246273-4db6-49c1-80d6-20db707698d0 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:14.227142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 946/1891] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:14 2022] POST /identity/v3/projects => generated 381 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:14.228489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0015a-f09e-45df-93a0-98800cece2e5 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:14.230476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-67f0015a-f09e-45df-93a0-98800cece2e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:14.238214 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 946/1892] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:14.240688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0246273-4db6-49c1-80d6-20db707698d0 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:14.244877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:14.245976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b0246273-4db6-49c1-80d6-20db707698d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:14.271573 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0246273-4db6-49c1-80d6-20db707698d0 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:14.333410 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0015a-f09e-45df-93a0-98800cece2e5 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:14.338695 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca1bc6f-7e0b-4c05-b604-9eefb8017629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:14.339343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1bc6f-7e0b-4c05-b604-9eefb8017629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:14.339787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1bc6f-7e0b-4c05-b604-9eefb8017629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:14.340066 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1bc6f-7e0b-4c05-b604-9eefb8017629 admin admin] PATH_INFO: `/v3/projects/53c2ec63e22e49e7adb8b3c85bd2a63c` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:14.341210 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca1bc6f-7e0b-4c05-b604-9eefb8017629 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53c2ec63e22e49e7adb8b3c85bd2a63c)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:14.343436 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca1bc6f-7e0b-4c05-b604-9eefb8017629 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:14.366172 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216eebc-811f-4380-af68-a067552ded52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:14.369170 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216eebc-811f-4380-af68-a067552ded52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:14.369467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216eebc-811f-4380-af68-a067552ded52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:14.369738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216eebc-811f-4380-af68-a067552ded52 admin admin] PATH_INFO: `/v3/users/4ed5c2a1c6a84caf92b213b9efd481e4` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:14.371056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216eebc-811f-4380-af68-a067552ded52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ed5c2a1c6a84caf92b213b9efd481e4)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:14.376944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216eebc-811f-4380-af68-a067552ded52 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:14.522082 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-eca1bc6f-7e0b-4c05-b604-9eefb8017629 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53c2ec63e22e49e7adb8b3c85bd2a63c', 'request_id': 'req-eca1bc6f-7e0b-4c05-b604-9eefb8017629'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:14.545623 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=86,keystone:DELETE=5,keystone:INSERT=9 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:14.617412 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-eca1bc6f-7e0b-4c05-b604-9eefb8017629 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53c2ec63e22e49e7adb8b3c85bd2a63c', 'request_id': 'req-eca1bc6f-7e0b-4c05-b604-9eefb8017629'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:14.625004 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-eca1bc6f-7e0b-4c05-b604-9eefb8017629 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53c2ec63e22e49e7adb8b3c85bd2a63c', 'request_id': 'req-eca1bc6f-7e0b-4c05-b604-9eefb8017629'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:14.639213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-eca1bc6f-7e0b-4c05-b604-9eefb8017629 admin admin] The token cache is being invalidate because project 53c2ec63e22e49e7adb8b3c85bd2a63c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:14.640138 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-eca1bc6f-7e0b-4c05-b604-9eefb8017629 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:14.641441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-eca1bc6f-7e0b-4c05-b604-9eefb8017629 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:14.646846 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 947/1893] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:14 2022] DELETE /identity/v3/projects/53c2ec63e22e49e7adb8b3c85bd2a63c => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:14.646846 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca1bc6f-7e0b-4c05-b604-9eefb8017629 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:14.646846 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-eca1bc6f-7e0b-4c05-b604-9eefb8017629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:14.651541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:14.668448 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4216eebc-811f-4380-af68-a067552ded52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ed5c2a1c6a84caf92b213b9efd481e4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:14.684912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4216eebc-811f-4380-af68-a067552ded52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ed5c2a1c6a84caf92b213b9efd481e4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:14.717110 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4216eebc-811f-4380-af68-a067552ded52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ed5c2a1c6a84caf92b213b9efd481e4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:14.739732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca1bc6f-7e0b-4c05-b604-9eefb8017629 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:14.745625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ed951-e34a-4cc2-bbb3-30ce83bea04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:14.746340 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ed951-e34a-4cc2-bbb3-30ce83bea04f admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:14.746623 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ed951-e34a-4cc2-bbb3-30ce83bea04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:14.746896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ed951-e34a-4cc2-bbb3-30ce83bea04f admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:14.747534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ed951-e34a-4cc2-bbb3-30ce83bea04f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:14.749576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ed951-e34a-4cc2-bbb3-30ce83bea04f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:14.829769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4216eebc-811f-4380-af68-a067552ded52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ed5c2a1c6a84caf92b213b9efd481e4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:14.830897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4216eebc-811f-4380-af68-a067552ded52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ed5c2a1c6a84caf92b213b9efd481e4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:14.842512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 947/1894] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:14 2022] DELETE /identity/v3/users/4ed5c2a1c6a84caf92b213b9efd481e4 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:14.844786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216eebc-811f-4380-af68-a067552ded52 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:14.847352 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4216eebc-811f-4380-af68-a067552ded52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:14.864574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 948/1895] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:14 2022] POST /identity/v3/users => generated 297 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:14.866403 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa782b-2880-436f-a790-4268af45c52f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:14.866823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa782b-2880-436f-a790-4268af45c52f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:14.867262 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa782b-2880-436f-a790-4268af45c52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:14.889264 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-09fa782b-2880-436f-a790-4268af45c52f None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:14.890694 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-09fa782b-2880-436f-a790-4268af45c52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:14.916949 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:14.949902 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216eebc-811f-4380-af68-a067552ded52 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:14.954790 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09b015-005a-4f06-ab17-463424163d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:14.955443 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09b015-005a-4f06-ab17-463424163d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:14.955726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09b015-005a-4f06-ab17-463424163d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:14.956002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09b015-005a-4f06-ab17-463424163d8d admin admin] PATH_INFO: `/v3/users/822b5ba1d3864908852d711222c6a72d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:14.957167 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09b015-005a-4f06-ab17-463424163d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=822b5ba1d3864908852d711222c6a72d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:14.959456 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09b015-005a-4f06-ab17-463424163d8d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:14.966193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 949/1896] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:14.967765 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44bf83-20d9-4fb3-8d5d-f4ec130f8bbb None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:14.968027 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44bf83-20d9-4fb3-8d5d-f4ec130f8bbb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:14.968301 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44bf83-20d9-4fb3-8d5d-f4ec130f8bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:14.990807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6a44bf83-20d9-4fb3-8d5d-f4ec130f8bbb None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:14.992063 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6a44bf83-20d9-4fb3-8d5d-f4ec130f8bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:15.001241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 950/1897] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:15.002011 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44bf83-20d9-4fb3-8d5d-f4ec130f8bbb None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:15.007451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:15.008719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6a44bf83-20d9-4fb3-8d5d-f4ec130f8bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:15.022747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44bf83-20d9-4fb3-8d5d-f4ec130f8bbb None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:15.028370 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365d20c-c904-4972-b014-de4f0a52c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:15.029173 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365d20c-c904-4972-b014-de4f0a52c3ad admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:15.029456 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365d20c-c904-4972-b014-de4f0a52c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:15.029738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365d20c-c904-4972-b014-de4f0a52c3ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:15.032867 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365d20c-c904-4972-b014-de4f0a52c3ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:15.032976 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365d20c-c904-4972-b014-de4f0a52c3ad admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:15.048164 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 951/1898] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:15.048164 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989a6dd-794d-488b-bfcb-b56909b285f0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:15.048164 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989a6dd-794d-488b-bfcb-b56909b285f0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:15.048164 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989a6dd-794d-488b-bfcb-b56909b285f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:15.071906 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9989a6dd-794d-488b-bfcb-b56909b285f0 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:15.073256 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9989a6dd-794d-488b-bfcb-b56909b285f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:15.082644 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 952/1899] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:15.083460 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989a6dd-794d-488b-bfcb-b56909b285f0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:15.086702 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9989a6dd-794d-488b-bfcb-b56909b285f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:15.165564 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989a6dd-794d-488b-bfcb-b56909b285f0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:15.170027 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba2980-fbce-4a24-8fc0-71564658a53e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:15.170701 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba2980-fbce-4a24-8fc0-71564658a53e service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:15.170985 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba2980-fbce-4a24-8fc0-71564658a53e service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:15.171257 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba2980-fbce-4a24-8fc0-71564658a53e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:15.181051 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba2980-fbce-4a24-8fc0-71564658a53e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:15.182599 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba2980-fbce-4a24-8fc0-71564658a53e service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:15.192468 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 953/1900] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:15.193408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba2980-fbce-4a24-8fc0-71564658a53e service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:15.203935 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba2980-fbce-4a24-8fc0-71564658a53e service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:15.208684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba88eb-ae0d-4240-bf53-efb977bd4e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:15.209370 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba88eb-ae0d-4240-bf53-efb977bd4e44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:15.209651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba88eb-ae0d-4240-bf53-efb977bd4e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:15.209964 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba88eb-ae0d-4240-bf53-efb977bd4e44 admin admin] PATH_INFO: `/v3/projects/9be7c44c77884a1a87e7f280b2f0c7e7/users/62f8c83b9c324fd6a4a8502a8ccc1b68/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:15.240029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba88eb-ae0d-4240-bf53-efb977bd4e44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9be7c44c77884a1a87e7f280b2f0c7e7, user_id=62f8c83b9c324fd6a4a8502a8ccc1b68, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:15.240283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-df09b015-005a-4f06-ab17-463424163d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '822b5ba1d3864908852d711222c6a72d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:15.244035 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba88eb-ae0d-4240-bf53-efb977bd4e44 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:15.261904 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-df09b015-005a-4f06-ab17-463424163d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '822b5ba1d3864908852d711222c6a72d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:15.271726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-df09b015-005a-4f06-ab17-463424163d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '822b5ba1d3864908852d711222c6a72d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:15.308318 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:15.340798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 954/1901] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:15 2022] PUT /identity/v3/projects/9be7c44c77884a1a87e7f280b2f0c7e7/users/62f8c83b9c324fd6a4a8502a8ccc1b68/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:15.342328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba88eb-ae0d-4240-bf53-efb977bd4e44 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:15.389757 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-df09b015-005a-4f06-ab17-463424163d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '822b5ba1d3864908852d711222c6a72d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:15.390569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-df09b015-005a-4f06-ab17-463424163d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '822b5ba1d3864908852d711222c6a72d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:15.392110 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:15.397933 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 948/1902] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:14 2022] DELETE /identity/v3/users/822b5ba1d3864908852d711222c6a72d => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:15.398802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09b015-005a-4f06-ab17-463424163d8d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:15.401648 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-df09b015-005a-4f06-ab17-463424163d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:15.429323 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba88eb-ae0d-4240-bf53-efb977bd4e44 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:15.487149 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09b015-005a-4f06-ab17-463424163d8d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:15.497475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0eb919-f05e-4d1f-8455-a368eb3823d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:15.498612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0eb919-f05e-4d1f-8455-a368eb3823d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:15.499413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0eb919-f05e-4d1f-8455-a368eb3823d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:15.500803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0eb919-f05e-4d1f-8455-a368eb3823d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:15.502970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0eb919-f05e-4d1f-8455-a368eb3823d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:15.505191 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0eb919-f05e-4d1f-8455-a368eb3823d8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:15.523774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 949/1903] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:15.525164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1e779-aa08-4533-8113-dab566e6745e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:15.525450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1e779-aa08-4533-8113-dab566e6745e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:15.525731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1e779-aa08-4533-8113-dab566e6745e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:15.533797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3286a-fd1a-417d-b60b-2c67134cc097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:15.534596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3286a-fd1a-417d-b60b-2c67134cc097 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:15.534984 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3286a-fd1a-417d-b60b-2c67134cc097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:15.535515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3286a-fd1a-417d-b60b-2c67134cc097 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:15.536305 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3286a-fd1a-417d-b60b-2c67134cc097 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:07:15.536737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3286a-fd1a-417d-b60b-2c67134cc097 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:15.539571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3286a-fd1a-417d-b60b-2c67134cc097 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:15.551524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d3c1e779-aa08-4533-8113-dab566e6745e None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:15.553893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d3c1e779-aa08-4533-8113-dab566e6745e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:15.557526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 955/1904] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:07:15.564548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3286a-fd1a-417d-b60b-2c67134cc097 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:15.567109 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 950/1905] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:15.572074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1e779-aa08-4533-8113-dab566e6745e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:15.572074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d3c1e779-aa08-4533-8113-dab566e6745e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:15.585679 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1e779-aa08-4533-8113-dab566e6745e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:15.591124 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284fea6-aa44-45fa-9221-5011088892f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:15.593090 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284fea6-aa44-45fa-9221-5011088892f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:15.593090 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284fea6-aa44-45fa-9221-5011088892f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:15.595789 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3286a-fd1a-417d-b60b-2c67134cc097 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:15.597145 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284fea6-aa44-45fa-9221-5011088892f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:15.597145 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284fea6-aa44-45fa-9221-5011088892f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:15.597145 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284fea6-aa44-45fa-9221-5011088892f9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:15.605706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008ab7d-f1fd-43ad-9c2b-db3c29c457a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:15.607097 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008ab7d-f1fd-43ad-9c2b-db3c29c457a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:15.607727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008ab7d-f1fd-43ad-9c2b-db3c29c457a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:15.608377 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008ab7d-f1fd-43ad-9c2b-db3c29c457a9 admin admin] PATH_INFO: `/v3/projects/9be7c44c77884a1a87e7f280b2f0c7e7/users/62f8c83b9c324fd6a4a8502a8ccc1b68/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:15.612707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008ab7d-f1fd-43ad-9c2b-db3c29c457a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9be7c44c77884a1a87e7f280b2f0c7e7, user_id=62f8c83b9c324fd6a4a8502a8ccc1b68, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:15.616728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008ab7d-f1fd-43ad-9c2b-db3c29c457a9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:15.632813 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 956/1906] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:15 2022] PUT /identity/v3/projects/9be7c44c77884a1a87e7f280b2f0c7e7/users/62f8c83b9c324fd6a4a8502a8ccc1b68/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:15.634393 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008ab7d-f1fd-43ad-9c2b-db3c29c457a9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:15.636208 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b008ab7d-f1fd-43ad-9c2b-db3c29c457a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:15.651573 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:15.695703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 951/1907] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:15 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:15.695703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bd72a-dd45-47ff-a244-3744720f4335 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:15.695703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bd72a-dd45-47ff-a244-3744720f4335 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:15.695703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bd72a-dd45-47ff-a244-3744720f4335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:15.713709 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:15.715770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008ab7d-f1fd-43ad-9c2b-db3c29c457a9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:15.735792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a83bd72a-dd45-47ff-a244-3744720f4335 None None] MFA Rules not processed for user `62f8c83b9c324fd6a4a8502a8ccc1b68`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:15.737320 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a83bd72a-dd45-47ff-a244-3744720f4335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:15.804346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af21bc-6198-4995-b427-eeb74c8efdb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:15.805773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af21bc-6198-4995-b427-eeb74c8efdb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:15.806558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af21bc-6198-4995-b427-eeb74c8efdb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:15.807281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af21bc-6198-4995-b427-eeb74c8efdb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:15.845909 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 952/1908] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:15 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:15.846774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83bd72a-dd45-47ff-a244-3744720f4335 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:15.896142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af21bc-6198-4995-b427-eeb74c8efdb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:15.897599 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af21bc-6198-4995-b427-eeb74c8efdb0 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:15.911111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 957/1909] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:15.914161 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af21bc-6198-4995-b427-eeb74c8efdb0 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:15.916432 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a3af21bc-6198-4995-b427-eeb74c8efdb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:15.943978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83bd72a-dd45-47ff-a244-3744720f4335 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:15.956442 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1631f9-eee7-4f84-bf59-511c7590f065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:15.956938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1631f9-eee7-4f84-bf59-511c7590f065 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:15.957270 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1631f9-eee7-4f84-bf59-511c7590f065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:15.958161 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1631f9-eee7-4f84-bf59-511c7590f065 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:15.959093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1631f9-eee7-4f84-bf59-511c7590f065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:15.961945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1631f9-eee7-4f84-bf59-511c7590f065 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:16.012523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af21bc-6198-4995-b427-eeb74c8efdb0 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:16.018819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdd7f3-9630-47f9-ac24-3bee94c65947 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:16.026031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdd7f3-9630-47f9-ac24-3bee94c65947 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:16.026031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdd7f3-9630-47f9-ac24-3bee94c65947 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:16.026031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdd7f3-9630-47f9-ac24-3bee94c65947 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:16.050393 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:16.054754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfdd7f3-9630-47f9-ac24-3bee94c65947 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:16.057575 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfdd7f3-9630-47f9-ac24-3bee94c65947 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:16.082493 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 958/1910] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:16.083342 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 953/1911] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:15 2022] POST /identity/v3/users => generated 291 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:16.089001 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdd7f3-9630-47f9-ac24-3bee94c65947 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:16.100673 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdd7f3-9630-47f9-ac24-3bee94c65947 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:16.116496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c887fb2-85a7-4cbd-9f47-9221c2e055da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:16.117150 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c887fb2-85a7-4cbd-9f47-9221c2e055da admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:16.117420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c887fb2-85a7-4cbd-9f47-9221c2e055da admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:16.117700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c887fb2-85a7-4cbd-9f47-9221c2e055da admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:16.118421 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c887fb2-85a7-4cbd-9f47-9221c2e055da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:16.120803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c887fb2-85a7-4cbd-9f47-9221c2e055da admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:16.137882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 959/1912] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:16.141667 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1631f9-eee7-4f84-bf59-511c7590f065 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:16.162634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1631f9-eee7-4f84-bf59-511c7590f065 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:16.175550 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f168bc-7ca5-4ef2-8487-1b546d3f59be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:16.176207 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f168bc-7ca5-4ef2-8487-1b546d3f59be admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:16.176497 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f168bc-7ca5-4ef2-8487-1b546d3f59be admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:16.176838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f168bc-7ca5-4ef2-8487-1b546d3f59be admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:16.177616 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f168bc-7ca5-4ef2-8487-1b546d3f59be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:16.179976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f168bc-7ca5-4ef2-8487-1b546d3f59be admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:16.200475 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 954/1913] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:16.200572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c887fb2-85a7-4cbd-9f47-9221c2e055da admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:16.201677 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7c887fb2-85a7-4cbd-9f47-9221c2e055da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:16.214826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f168bc-7ca5-4ef2-8487-1b546d3f59be admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:16.228478 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f168bc-7ca5-4ef2-8487-1b546d3f59be admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:16.268597 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8df18-c085-400b-b457-ad070b26b281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:16.270835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8df18-c085-400b-b457-ad070b26b281 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:16.271396 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8df18-c085-400b-b457-ad070b26b281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:16.272029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8df18-c085-400b-b457-ad070b26b281 admin admin] PATH_INFO: `/v3/projects/9adc0a5801cf49d8a7ce8d1983743a4c/users/730a9fe071824d4799e46051db0797de/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:16.288437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c887fb2-85a7-4cbd-9f47-9221c2e055da admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:16.308713 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937be69-81ed-498c-9a9f-bedb39e925ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:16.309395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937be69-81ed-498c-9a9f-bedb39e925ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:16.309894 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937be69-81ed-498c-9a9f-bedb39e925ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:16.310242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937be69-81ed-498c-9a9f-bedb39e925ce admin admin] PATH_INFO: `/v3/projects/c49e18812eae47d4977865566fb64f1d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:16.316250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937be69-81ed-498c-9a9f-bedb39e925ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=c49e18812eae47d4977865566fb64f1d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:16.319145 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937be69-81ed-498c-9a9f-bedb39e925ce admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:16.344961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8df18-c085-400b-b457-ad070b26b281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9adc0a5801cf49d8a7ce8d1983743a4c, user_id=730a9fe071824d4799e46051db0797de, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:16.349464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8df18-c085-400b-b457-ad070b26b281 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:16.477119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 955/1914] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:16 2022] PUT /identity/v3/projects/9adc0a5801cf49d8a7ce8d1983743a4c/users/730a9fe071824d4799e46051db0797de/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:16.489101 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0651d-4a4c-4175-99cb-6c483617c1d6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:16.489893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0651d-4a4c-4175-99cb-6c483617c1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:16.490332 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0651d-4a4c-4175-99cb-6c483617c1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:16.520979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b937be69-81ed-498c-9a9f-bedb39e925ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c49e18812eae47d4977865566fb64f1d', 'request_id': 'req-b937be69-81ed-498c-9a9f-bedb39e925ce'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:16.536745 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-24a0651d-4a4c-4175-99cb-6c483617c1d6 None None] MFA Rules not processed for user `730a9fe071824d4799e46051db0797de`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:16.536745 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-24a0651d-4a4c-4175-99cb-6c483617c1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:16.573379 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:16.606982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:16.625733 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 956/1915] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:16.626921 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0651d-4a4c-4175-99cb-6c483617c1d6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:16.632181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-24a0651d-4a4c-4175-99cb-6c483617c1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:16.647981 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b937be69-81ed-498c-9a9f-bedb39e925ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c49e18812eae47d4977865566fb64f1d', 'request_id': 'req-b937be69-81ed-498c-9a9f-bedb39e925ce'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:16.658594 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b937be69-81ed-498c-9a9f-bedb39e925ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c49e18812eae47d4977865566fb64f1d', 'request_id': 'req-b937be69-81ed-498c-9a9f-bedb39e925ce'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:16.677781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b937be69-81ed-498c-9a9f-bedb39e925ce admin admin] The token cache is being invalidate because project c49e18812eae47d4977865566fb64f1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:16.678334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b937be69-81ed-498c-9a9f-bedb39e925ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:16.678933 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b937be69-81ed-498c-9a9f-bedb39e925ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:16.680595 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 960/1916] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:16 2022] DELETE /identity/v3/projects/c49e18812eae47d4977865566fb64f1d => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:16.683856 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937be69-81ed-498c-9a9f-bedb39e925ce admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:16.683856 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b937be69-81ed-498c-9a9f-bedb39e925ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:16.735493 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0651d-4a4c-4175-99cb-6c483617c1d6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:16.741250 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf5e20-3ec1-4b10-98b9-cb4113e0aada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:16.741963 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf5e20-3ec1-4b10-98b9-cb4113e0aada admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:16.742251 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf5e20-3ec1-4b10-98b9-cb4113e0aada admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:16.742702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf5e20-3ec1-4b10-98b9-cb4113e0aada admin admin] PATH_INFO: `/v3/projects/1c942433245d423199cc295d416bbbcc` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:16.744612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf5e20-3ec1-4b10-98b9-cb4113e0aada admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c942433245d423199cc295d416bbbcc)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:16.746908 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf5e20-3ec1-4b10-98b9-cb4113e0aada admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:16.782189 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937be69-81ed-498c-9a9f-bedb39e925ce admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:16.787090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482fd10-ef56-4e9b-8de9-7cb1aa87e41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:16.792766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482fd10-ef56-4e9b-8de9-7cb1aa87e41f admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:16.792766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482fd10-ef56-4e9b-8de9-7cb1aa87e41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:16.792766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482fd10-ef56-4e9b-8de9-7cb1aa87e41f admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:16.792766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6482fd10-ef56-4e9b-8de9-7cb1aa87e41f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:16.792766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6482fd10-ef56-4e9b-8de9-7cb1aa87e41f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:16.900688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5cdf5e20-3ec1-4b10-98b9-cb4113e0aada admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c942433245d423199cc295d416bbbcc', 'request_id': 'req-5cdf5e20-3ec1-4b10-98b9-cb4113e0aada'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:16.924575 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 961/1917] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:16 2022] POST /identity/v3/projects => generated 381 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:16.924575 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482fd10-ef56-4e9b-8de9-7cb1aa87e41f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:16.924575 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6482fd10-ef56-4e9b-8de9-7cb1aa87e41f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:16.993515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482fd10-ef56-4e9b-8de9-7cb1aa87e41f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:16.998591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20e8e6-9e8b-4c5a-b304-1b7db2dc08af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:16.999382 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20e8e6-9e8b-4c5a-b304-1b7db2dc08af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:16.999713 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20e8e6-9e8b-4c5a-b304-1b7db2dc08af admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:17.000719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20e8e6-9e8b-4c5a-b304-1b7db2dc08af admin admin] PATH_INFO: `/v3/users/91a9ec8d71824332838b3360bc90f800` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:17.001932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5cdf5e20-3ec1-4b10-98b9-cb4113e0aada admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c942433245d423199cc295d416bbbcc', 'request_id': 'req-5cdf5e20-3ec1-4b10-98b9-cb4113e0aada'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:17.006655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20e8e6-9e8b-4c5a-b304-1b7db2dc08af admin admin] RBAC: Authorizing `identity:delete_user(user_id=91a9ec8d71824332838b3360bc90f800)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:17.008388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5cdf5e20-3ec1-4b10-98b9-cb4113e0aada admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c942433245d423199cc295d416bbbcc', 'request_id': 'req-5cdf5e20-3ec1-4b10-98b9-cb4113e0aada'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:17.010923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20e8e6-9e8b-4c5a-b304-1b7db2dc08af admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:17.021550 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5cdf5e20-3ec1-4b10-98b9-cb4113e0aada admin admin] The token cache is being invalidate because project 1c942433245d423199cc295d416bbbcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:17.021837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5cdf5e20-3ec1-4b10-98b9-cb4113e0aada admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:17.022825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5cdf5e20-3ec1-4b10-98b9-cb4113e0aada admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:17.024953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 957/1918] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:16 2022] DELETE /identity/v3/projects/1c942433245d423199cc295d416bbbcc => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:17.028173 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf5e20-3ec1-4b10-98b9-cb4113e0aada admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:17.029058 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5cdf5e20-3ec1-4b10-98b9-cb4113e0aada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:17.122132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf5e20-3ec1-4b10-98b9-cb4113e0aada admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:17.128383 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f78d22-1c6b-4bcb-96d9-40837ff31169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:17.129063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f78d22-1c6b-4bcb-96d9-40837ff31169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:17.129441 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f78d22-1c6b-4bcb-96d9-40837ff31169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:17.129905 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f78d22-1c6b-4bcb-96d9-40837ff31169 admin admin] PATH_INFO: `/v3/users/ba970bd70bd441d798f3048aa9520fd1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:17.131423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f78d22-1c6b-4bcb-96d9-40837ff31169 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba970bd70bd441d798f3048aa9520fd1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:17.133607 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f78d22-1c6b-4bcb-96d9-40837ff31169 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:17.212509 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:17.270117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4b20e8e6-9e8b-4c5a-b304-1b7db2dc08af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91a9ec8d71824332838b3360bc90f800'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:17.282754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:17.287980 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4b20e8e6-9e8b-4c5a-b304-1b7db2dc08af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91a9ec8d71824332838b3360bc90f800'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:17.297626 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4b20e8e6-9e8b-4c5a-b304-1b7db2dc08af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91a9ec8d71824332838b3360bc90f800'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:17.365175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d3f78d22-1c6b-4bcb-96d9-40837ff31169 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba970bd70bd441d798f3048aa9520fd1'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:17.381758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d3f78d22-1c6b-4bcb-96d9-40837ff31169 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba970bd70bd441d798f3048aa9520fd1'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:17.389657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4b20e8e6-9e8b-4c5a-b304-1b7db2dc08af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91a9ec8d71824332838b3360bc90f800'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:17.391250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4b20e8e6-9e8b-4c5a-b304-1b7db2dc08af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91a9ec8d71824332838b3360bc90f800'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:17.393025 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d3f78d22-1c6b-4bcb-96d9-40837ff31169 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba970bd70bd441d798f3048aa9520fd1'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:17.402658 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 962/1919] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:16 2022] DELETE /identity/v3/users/91a9ec8d71824332838b3360bc90f800 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:17.404033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20e8e6-9e8b-4c5a-b304-1b7db2dc08af admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:17.409351 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4b20e8e6-9e8b-4c5a-b304-1b7db2dc08af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:17.486894 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d3f78d22-1c6b-4bcb-96d9-40837ff31169 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba970bd70bd441d798f3048aa9520fd1'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:17.487854 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d3f78d22-1c6b-4bcb-96d9-40837ff31169 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba970bd70bd441d798f3048aa9520fd1'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:17.498279 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 958/1920] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:17 2022] DELETE /identity/v3/users/ba970bd70bd441d798f3048aa9520fd1 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:17.499463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f78d22-1c6b-4bcb-96d9-40837ff31169 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:17.502621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d3f78d22-1c6b-4bcb-96d9-40837ff31169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:17.513377 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20e8e6-9e8b-4c5a-b304-1b7db2dc08af admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:17.603520 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f78d22-1c6b-4bcb-96d9-40837ff31169 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:17.610350 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35156bec-0a79-4fc3-ac4b-29cb9ec5abba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:17.611445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35156bec-0a79-4fc3-ac4b-29cb9ec5abba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:17.611889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35156bec-0a79-4fc3-ac4b-29cb9ec5abba admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:17.612469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35156bec-0a79-4fc3-ac4b-29cb9ec5abba admin admin] PATH_INFO: `/v3/users/e74fc6c7f0b84e0aa152aee992e81378` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:17.614559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35156bec-0a79-4fc3-ac4b-29cb9ec5abba admin admin] RBAC: Authorizing `identity:delete_user(user_id=e74fc6c7f0b84e0aa152aee992e81378)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:17.615266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5867dea-b6f4-464e-80f0-b82864c96d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:17.616010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5867dea-b6f4-464e-80f0-b82864c96d24 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:17.616395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5867dea-b6f4-464e-80f0-b82864c96d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:17.616772 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5867dea-b6f4-464e-80f0-b82864c96d24 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:17.617925 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5867dea-b6f4-464e-80f0-b82864c96d24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:17.618169 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35156bec-0a79-4fc3-ac4b-29cb9ec5abba admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:17.620914 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5867dea-b6f4-464e-80f0-b82864c96d24 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:17.742412 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 963/1921] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:17 2022] POST /identity/v3/users => generated 297 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:17.743676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4dba65-5d05-4703-ba75-02df2a842211 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:17.743964 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4dba65-5d05-4703-ba75-02df2a842211 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:17.744246 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4dba65-5d05-4703-ba75-02df2a842211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:17.766775 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-5b4dba65-5d05-4703-ba75-02df2a842211 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:17.766775 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5b4dba65-5d05-4703-ba75-02df2a842211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:17.781019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 964/1922] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:17.787912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4dba65-5d05-4703-ba75-02df2a842211 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:17.787912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5b4dba65-5d05-4703-ba75-02df2a842211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:17.799796 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4dba65-5d05-4703-ba75-02df2a842211 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:17.805384 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd374eb-8e8b-471e-b050-d27e4d3ae88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:17.809804 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd374eb-8e8b-471e-b050-d27e4d3ae88e admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:17.809804 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd374eb-8e8b-471e-b050-d27e4d3ae88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:17.809804 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd374eb-8e8b-471e-b050-d27e4d3ae88e admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:17.809804 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd374eb-8e8b-471e-b050-d27e4d3ae88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:17.809804 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd374eb-8e8b-471e-b050-d27e4d3ae88e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:17.827517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 965/1923] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:17.828715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd374eb-8e8b-471e-b050-d27e4d3ae88e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:17.831439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fdd374eb-8e8b-471e-b050-d27e4d3ae88e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:17.910982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-35156bec-0a79-4fc3-ac4b-29cb9ec5abba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e74fc6c7f0b84e0aa152aee992e81378'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:17.933472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd374eb-8e8b-471e-b050-d27e4d3ae88e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:17.939705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd46d7-b90a-43fc-992d-757092697b95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:17.940565 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd46d7-b90a-43fc-992d-757092697b95 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:17.941363 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd46d7-b90a-43fc-992d-757092697b95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:17.941526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd46d7-b90a-43fc-992d-757092697b95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:17.954643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-35156bec-0a79-4fc3-ac4b-29cb9ec5abba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e74fc6c7f0b84e0aa152aee992e81378'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:17.964702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-35156bec-0a79-4fc3-ac4b-29cb9ec5abba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e74fc6c7f0b84e0aa152aee992e81378'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:17.965769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd46d7-b90a-43fc-992d-757092697b95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:17.970435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd46d7-b90a-43fc-992d-757092697b95 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:17.981961 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 966/1924] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:17.983028 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd46d7-b90a-43fc-992d-757092697b95 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:17.998713 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd46d7-b90a-43fc-992d-757092697b95 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:18.014851 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26cecb-cc89-40e9-b94f-f1e005b95ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:18.015781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26cecb-cc89-40e9-b94f-f1e005b95ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:18.018403 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26cecb-cc89-40e9-b94f-f1e005b95ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:18.019358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26cecb-cc89-40e9-b94f-f1e005b95ad1 admin admin] PATH_INFO: `/v3/projects/60a8e23a1a3147d3a6efd25d1f3653b5/users/a81b4918dd4043a0846367084614c083/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:18.053600 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26cecb-cc89-40e9-b94f-f1e005b95ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a8e23a1a3147d3a6efd25d1f3653b5, user_id=a81b4918dd4043a0846367084614c083, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:18.056549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26cecb-cc89-40e9-b94f-f1e005b95ad1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:18.091275 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-35156bec-0a79-4fc3-ac4b-29cb9ec5abba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e74fc6c7f0b84e0aa152aee992e81378'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:18.091917 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-35156bec-0a79-4fc3-ac4b-29cb9ec5abba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e74fc6c7f0b84e0aa152aee992e81378'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:18.101774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 959/1925] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:17 2022] DELETE /identity/v3/users/e74fc6c7f0b84e0aa152aee992e81378 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:18.144117 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35156bec-0a79-4fc3-ac4b-29cb9ec5abba admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:18.145664 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-35156bec-0a79-4fc3-ac4b-29cb9ec5abba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:18.212413 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 967/1926] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:17 2022] PUT /identity/v3/projects/60a8e23a1a3147d3a6efd25d1f3653b5/users/a81b4918dd4043a0846367084614c083/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:18.218962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26cecb-cc89-40e9-b94f-f1e005b95ad1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:18.222592 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3e26cecb-cc89-40e9-b94f-f1e005b95ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:18.267506 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35156bec-0a79-4fc3-ac4b-29cb9ec5abba admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:18.335651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26cecb-cc89-40e9-b94f-f1e005b95ad1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:18.339883 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfbfd5-d956-470c-8fe5-279b0b7bd846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:18.340608 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfbfd5-d956-470c-8fe5-279b0b7bd846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:18.340821 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfbfd5-d956-470c-8fe5-279b0b7bd846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:18.341104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfbfd5-d956-470c-8fe5-279b0b7bd846 admin admin] PATH_INFO: `/v3/projects/5770f7450c71491e9b94ce392e5489ea` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:18.342697 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bfbfd5-d956-470c-8fe5-279b0b7bd846 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5770f7450c71491e9b94ce392e5489ea)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:18.343090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768169e8-7faa-4722-9255-190733d849ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:18.344729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768169e8-7faa-4722-9255-190733d849ad admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:18.345021 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768169e8-7faa-4722-9255-190733d849ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:18.345186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bfbfd5-d956-470c-8fe5-279b0b7bd846 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:18.345286 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768169e8-7faa-4722-9255-190733d849ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:18.346067 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768169e8-7faa-4722-9255-190733d849ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:18.349016 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768169e8-7faa-4722-9255-190733d849ad admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:18.368145 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 968/1927] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:18.375655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768169e8-7faa-4722-9255-190733d849ad admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:18.387342 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768169e8-7faa-4722-9255-190733d849ad admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:18.403643 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33270682-8bfc-47a7-8282-040854ca1e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:18.407024 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33270682-8bfc-47a7-8282-040854ca1e96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:18.407706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33270682-8bfc-47a7-8282-040854ca1e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:18.408862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33270682-8bfc-47a7-8282-040854ca1e96 admin admin] PATH_INFO: `/v3/projects/60a8e23a1a3147d3a6efd25d1f3653b5/users/a81b4918dd4043a0846367084614c083/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:18.412709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33270682-8bfc-47a7-8282-040854ca1e96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a8e23a1a3147d3a6efd25d1f3653b5, user_id=a81b4918dd4043a0846367084614c083, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:18.415865 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33270682-8bfc-47a7-8282-040854ca1e96 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:18.439534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 969/1928] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:18 2022] PUT /identity/v3/projects/60a8e23a1a3147d3a6efd25d1f3653b5/users/a81b4918dd4043a0846367084614c083/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:18.459914 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914f6d4-78d1-48ad-b73a-bbf81bc63f1c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:18.460269 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914f6d4-78d1-48ad-b73a-bbf81bc63f1c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:18.460753 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914f6d4-78d1-48ad-b73a-bbf81bc63f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:18.522823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8914f6d4-78d1-48ad-b73a-bbf81bc63f1c None None] MFA Rules not processed for user `a81b4918dd4043a0846367084614c083`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:18.525525 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8914f6d4-78d1-48ad-b73a-bbf81bc63f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:18.528583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a4bfbfd5-d956-470c-8fe5-279b0b7bd846 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5770f7450c71491e9b94ce392e5489ea', 'request_id': 'req-a4bfbfd5-d956-470c-8fe5-279b0b7bd846'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:18.612834 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a4bfbfd5-d956-470c-8fe5-279b0b7bd846 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5770f7450c71491e9b94ce392e5489ea', 'request_id': 'req-a4bfbfd5-d956-470c-8fe5-279b0b7bd846'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:18.626997 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a4bfbfd5-d956-470c-8fe5-279b0b7bd846 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5770f7450c71491e9b94ce392e5489ea', 'request_id': 'req-a4bfbfd5-d956-470c-8fe5-279b0b7bd846'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:18.635042 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 970/1929] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:18.646379 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:18.654548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a4bfbfd5-d956-470c-8fe5-279b0b7bd846 admin admin] The token cache is being invalidate because project 5770f7450c71491e9b94ce392e5489ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:18.655489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914f6d4-78d1-48ad-b73a-bbf81bc63f1c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:18.657491 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a4bfbfd5-d956-470c-8fe5-279b0b7bd846 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:18.658172 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a4bfbfd5-d956-470c-8fe5-279b0b7bd846 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:18.660209 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 960/1930] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:18 2022] DELETE /identity/v3/projects/5770f7450c71491e9b94ce392e5489ea => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:07:18.662729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8914f6d4-78d1-48ad-b73a-bbf81bc63f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:18.670860 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044791c-b9db-4edb-9a06-14b487ad7671 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:18.671174 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044791c-b9db-4edb-9a06-14b487ad7671 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:18.672680 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044791c-b9db-4edb-9a06-14b487ad7671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:18.701986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d044791c-b9db-4edb-9a06-14b487ad7671 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:18.705037 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d044791c-b9db-4edb-9a06-14b487ad7671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:18.763352 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:18.772432 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914f6d4-78d1-48ad-b73a-bbf81bc63f1c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:18.781097 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146011f5-e966-4135-91bd-ebd32bcb3ced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:18.783056 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146011f5-e966-4135-91bd-ebd32bcb3ced service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:18.784642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146011f5-e966-4135-91bd-ebd32bcb3ced service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:18.785089 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146011f5-e966-4135-91bd-ebd32bcb3ced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:18.788437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-146011f5-e966-4135-91bd-ebd32bcb3ced service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:18.810079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 961/1931] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:18.813399 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146011f5-e966-4135-91bd-ebd32bcb3ced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:18.816591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146011f5-e966-4135-91bd-ebd32bcb3ced service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:18.822638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8835e27-4d02-4f32-8ea5-7297a72e9766 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:18.827949 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8835e27-4d02-4f32-8ea5-7297a72e9766 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:18.827949 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8835e27-4d02-4f32-8ea5-7297a72e9766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:18.838258 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 971/1932] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:18.872376 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a8835e27-4d02-4f32-8ea5-7297a72e9766 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:18.874032 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a8835e27-4d02-4f32-8ea5-7297a72e9766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:18.893990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 962/1933] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:18.896315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146011f5-e966-4135-91bd-ebd32bcb3ced service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:18.931763 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146011f5-e966-4135-91bd-ebd32bcb3ced service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:18.943408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be92ac-dad1-4eef-9ecb-c939f0620db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:18.943408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be92ac-dad1-4eef-9ecb-c939f0620db4 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:18.943408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be92ac-dad1-4eef-9ecb-c939f0620db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:18.944231 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be92ac-dad1-4eef-9ecb-c939f0620db4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:18.945769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be92ac-dad1-4eef-9ecb-c939f0620db4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:07:18.946344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be92ac-dad1-4eef-9ecb-c939f0620db4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:18.949982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be92ac-dad1-4eef-9ecb-c939f0620db4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:19.026573 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8835e27-4d02-4f32-8ea5-7297a72e9766 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:19.030072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 972/1934] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:07:19.045544 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8835e27-4d02-4f32-8ea5-7297a72e9766 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:19.052870 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd3dcff-57ad-4434-824a-4781a2cee89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:19.053831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3dcff-57ad-4434-824a-4781a2cee89c admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:19.054321 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3dcff-57ad-4434-824a-4781a2cee89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:19.054743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3dcff-57ad-4434-824a-4781a2cee89c admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:19.055576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd3dcff-57ad-4434-824a-4781a2cee89c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:19.057844 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd3dcff-57ad-4434-824a-4781a2cee89c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:19.081038 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be92ac-dad1-4eef-9ecb-c939f0620db4 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:19.082040 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-25be92ac-dad1-4eef-9ecb-c939f0620db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:19.105896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be92ac-dad1-4eef-9ecb-c939f0620db4 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:19.113804 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67b346-5340-4101-bf60-9f74bcd5d093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:19.114756 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67b346-5340-4101-bf60-9f74bcd5d093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:19.115332 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67b346-5340-4101-bf60-9f74bcd5d093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:19.115899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67b346-5340-4101-bf60-9f74bcd5d093 admin admin] PATH_INFO: `/v3/projects/528295fa52bf4f5fa8f0bfd56db158d7` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:19.123564 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67b346-5340-4101-bf60-9f74bcd5d093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=528295fa52bf4f5fa8f0bfd56db158d7)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:19.132474 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67b346-5340-4101-bf60-9f74bcd5d093 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:19.210624 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 963/1935] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:19 2022] POST /identity/v3/projects => generated 409 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:19.212116 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd3dcff-57ad-4434-824a-4781a2cee89c admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:19.218956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4bd3dcff-57ad-4434-824a-4781a2cee89c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:19.322436 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd3dcff-57ad-4434-824a-4781a2cee89c admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:19.327871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11ea85-1314-4d6a-8b99-fdf1e6c68e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:19.329357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11ea85-1314-4d6a-8b99-fdf1e6c68e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:19.330042 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11ea85-1314-4d6a-8b99-fdf1e6c68e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:19.330414 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11ea85-1314-4d6a-8b99-fdf1e6c68e06 admin admin] PATH_INFO: `/v3/projects/344d17db4b3a4f329e9fd1fd7eb6a2b1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:19.331734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11ea85-1314-4d6a-8b99-fdf1e6c68e06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=344d17db4b3a4f329e9fd1fd7eb6a2b1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:19.335230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11ea85-1314-4d6a-8b99-fdf1e6c68e06 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:19.398029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-da67b346-5340-4101-bf60-9f74bcd5d093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '528295fa52bf4f5fa8f0bfd56db158d7', 'request_id': 'req-da67b346-5340-4101-bf60-9f74bcd5d093'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:19.480401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-da67b346-5340-4101-bf60-9f74bcd5d093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '528295fa52bf4f5fa8f0bfd56db158d7', 'request_id': 'req-da67b346-5340-4101-bf60-9f74bcd5d093'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:19.496948 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-da67b346-5340-4101-bf60-9f74bcd5d093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '528295fa52bf4f5fa8f0bfd56db158d7', 'request_id': 'req-da67b346-5340-4101-bf60-9f74bcd5d093'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:19.505187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6e11ea85-1314-4d6a-8b99-fdf1e6c68e06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '344d17db4b3a4f329e9fd1fd7eb6a2b1', 'request_id': 'req-6e11ea85-1314-4d6a-8b99-fdf1e6c68e06'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:19.518324 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-da67b346-5340-4101-bf60-9f74bcd5d093 admin admin] The token cache is being invalidate because project 528295fa52bf4f5fa8f0bfd56db158d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:19.518615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-da67b346-5340-4101-bf60-9f74bcd5d093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:19.519513 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-da67b346-5340-4101-bf60-9f74bcd5d093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:19.523881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 973/1936] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:19 2022] DELETE /identity/v3/projects/528295fa52bf4f5fa8f0bfd56db158d7 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:07:19.523881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67b346-5340-4101-bf60-9f74bcd5d093 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:19.523881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-da67b346-5340-4101-bf60-9f74bcd5d093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:19.592652 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6e11ea85-1314-4d6a-8b99-fdf1e6c68e06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '344d17db4b3a4f329e9fd1fd7eb6a2b1', 'request_id': 'req-6e11ea85-1314-4d6a-8b99-fdf1e6c68e06'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:19.608417 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6e11ea85-1314-4d6a-8b99-fdf1e6c68e06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '344d17db4b3a4f329e9fd1fd7eb6a2b1', 'request_id': 'req-6e11ea85-1314-4d6a-8b99-fdf1e6c68e06'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:19.610010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67b346-5340-4101-bf60-9f74bcd5d093 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:19.626489 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6e11ea85-1314-4d6a-8b99-fdf1e6c68e06 admin admin] The token cache is being invalidate because project 344d17db4b3a4f329e9fd1fd7eb6a2b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:19.627186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6e11ea85-1314-4d6a-8b99-fdf1e6c68e06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:19.627790 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6e11ea85-1314-4d6a-8b99-fdf1e6c68e06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:19.629509 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 964/1937] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:19 2022] DELETE /identity/v3/projects/344d17db4b3a4f329e9fd1fd7eb6a2b1 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:19.630399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11ea85-1314-4d6a-8b99-fdf1e6c68e06 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:19.634146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6e11ea85-1314-4d6a-8b99-fdf1e6c68e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:19.706526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c528f-9eb2-4f32-8e0a-9d0581624118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:19.707778 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c528f-9eb2-4f32-8e0a-9d0581624118 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:19.709152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c528f-9eb2-4f32-8e0a-9d0581624118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:19.709605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c528f-9eb2-4f32-8e0a-9d0581624118 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:19.710510 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c528f-9eb2-4f32-8e0a-9d0581624118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:19.714091 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c528f-9eb2-4f32-8e0a-9d0581624118 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:19.735109 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11ea85-1314-4d6a-8b99-fdf1e6c68e06 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:19.742855 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ad628-251c-412f-b489-c8a5fc184984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:19.743634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ad628-251c-412f-b489-c8a5fc184984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:19.743927 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ad628-251c-412f-b489-c8a5fc184984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:19.744274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ad628-251c-412f-b489-c8a5fc184984 admin admin] PATH_INFO: `/v3/users/62f8c83b9c324fd6a4a8502a8ccc1b68` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:19.745486 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ad628-251c-412f-b489-c8a5fc184984 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62f8c83b9c324fd6a4a8502a8ccc1b68)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:19.747486 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ad628-251c-412f-b489-c8a5fc184984 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:19.865358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 974/1938] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:19 2022] POST /identity/v3/users => generated 305 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:19.867429 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8845fd-f854-494f-86e0-828e1327ce45 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:19.869194 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8845fd-f854-494f-86e0-828e1327ce45 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:19.869672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8845fd-f854-494f-86e0-828e1327ce45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:19.902223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3f8845fd-f854-494f-86e0-828e1327ce45 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:19.903537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3f8845fd-f854-494f-86e0-828e1327ce45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:19.913110 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 975/1939] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:19.913877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8845fd-f854-494f-86e0-828e1327ce45 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:19.915327 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3f8845fd-f854-494f-86e0-828e1327ce45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:19.927244 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8845fd-f854-494f-86e0-828e1327ce45 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:19.932708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9125ff-17e7-458b-9d1b-7c727b4a316d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:19.933316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9125ff-17e7-458b-9d1b-7c727b4a316d admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:19.933603 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9125ff-17e7-458b-9d1b-7c727b4a316d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:19.933910 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9125ff-17e7-458b-9d1b-7c727b4a316d admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:19.934559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9125ff-17e7-458b-9d1b-7c727b4a316d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:19.943278 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9125ff-17e7-458b-9d1b-7c727b4a316d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:19.947048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 976/1940] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:19.948113 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b264a9-a42d-47e1-878b-88a61f081eb1 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:19.948404 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b264a9-a42d-47e1-878b-88a61f081eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:19.948713 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b264a9-a42d-47e1-878b-88a61f081eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:19.968738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f0b264a9-a42d-47e1-878b-88a61f081eb1 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:19.969990 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f0b264a9-a42d-47e1-878b-88a61f081eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:19.978401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 977/1941] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:19.979040 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b264a9-a42d-47e1-878b-88a61f081eb1 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:19.988865 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b264a9-a42d-47e1-878b-88a61f081eb1 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:19.994639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a4079-40a7-48bb-8560-3c42eb886a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:19.995320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a4079-40a7-48bb-8560-3c42eb886a17 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:19.995536 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a4079-40a7-48bb-8560-3c42eb886a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:19.995811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a4079-40a7-48bb-8560-3c42eb886a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:19.996494 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a4079-40a7-48bb-8560-3c42eb886a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:19.998162 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a4079-40a7-48bb-8560-3c42eb886a17 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:20.009676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 978/1942] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:20.010751 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a4079-40a7-48bb-8560-3c42eb886a17 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:20.026790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a4079-40a7-48bb-8560-3c42eb886a17 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:20.033574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565daf0-f383-4f28-b926-e1391ff9c899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:20.034280 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565daf0-f383-4f28-b926-e1391ff9c899 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:20.034569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565daf0-f383-4f28-b926-e1391ff9c899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:20.034849 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565daf0-f383-4f28-b926-e1391ff9c899 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:20.035713 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565daf0-f383-4f28-b926-e1391ff9c899 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:07:20.036032 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565daf0-f383-4f28-b926-e1391ff9c899 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:20.036469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-f36ad628-251c-412f-b489-c8a5fc184984 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62f8c83b9c324fd6a4a8502a8ccc1b68'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:20.037920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565daf0-f383-4f28-b926-e1391ff9c899 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:20.055882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 979/1943] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:07:20.056977 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565daf0-f383-4f28-b926-e1391ff9c899 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:20.057977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-f36ad628-251c-412f-b489-c8a5fc184984 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62f8c83b9c324fd6a4a8502a8ccc1b68'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:20.070621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-f36ad628-251c-412f-b489-c8a5fc184984 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62f8c83b9c324fd6a4a8502a8ccc1b68'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:20.072683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565daf0-f383-4f28-b926-e1391ff9c899 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:20.083174 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a3800-3024-4371-b229-58377928de11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:20.083923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a3800-3024-4371-b229-58377928de11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:20.084297 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a3800-3024-4371-b229-58377928de11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:20.085255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a3800-3024-4371-b229-58377928de11 admin admin] PATH_INFO: `/v3/projects/7c5e7090d0c8426eb82d152a49029509/users/6297f7c6455c4a0e884453e4c422034a/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:20.102761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a3800-3024-4371-b229-58377928de11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c5e7090d0c8426eb82d152a49029509, user_id=6297f7c6455c4a0e884453e4c422034a, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:20.105597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a3800-3024-4371-b229-58377928de11 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:20.127110 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-f36ad628-251c-412f-b489-c8a5fc184984 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62f8c83b9c324fd6a4a8502a8ccc1b68'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:20.127768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-f36ad628-251c-412f-b489-c8a5fc184984 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62f8c83b9c324fd6a4a8502a8ccc1b68'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:20.138205 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 965/1944] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:19 2022] DELETE /identity/v3/users/62f8c83b9c324fd6a4a8502a8ccc1b68 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:20.152543 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ad628-251c-412f-b489-c8a5fc184984 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:20.158128 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f36ad628-251c-412f-b489-c8a5fc184984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:20.193893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 980/1945] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:20 2022] PUT /identity/v3/projects/7c5e7090d0c8426eb82d152a49029509/users/6297f7c6455c4a0e884453e4c422034a/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:20.214031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9a0e3-93d7-4a83-9ae3-dc2963af2082 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:20.214031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9a0e3-93d7-4a83-9ae3-dc2963af2082 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:20.214031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9a0e3-93d7-4a83-9ae3-dc2963af2082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:20.257574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-97a9a0e3-93d7-4a83-9ae3-dc2963af2082 None None] MFA Rules not processed for user `6297f7c6455c4a0e884453e4c422034a`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:20.259634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-97a9a0e3-93d7-4a83-9ae3-dc2963af2082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:20.349965 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ad628-251c-412f-b489-c8a5fc184984 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:20.371724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 981/1946] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:20 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:20.375572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9a0e3-93d7-4a83-9ae3-dc2963af2082 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:20.375572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-97a9a0e3-93d7-4a83-9ae3-dc2963af2082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:20.397224 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:20.417445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ddac0-00d9-4f34-b42e-c3003116a5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:20.418103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ddac0-00d9-4f34-b42e-c3003116a5b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:20.418417 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ddac0-00d9-4f34-b42e-c3003116a5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:20.418723 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ddac0-00d9-4f34-b42e-c3003116a5b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:20.419373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ddac0-00d9-4f34-b42e-c3003116a5b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:20.421384 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ddac0-00d9-4f34-b42e-c3003116a5b5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:20.450233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9a0e3-93d7-4a83-9ae3-dc2963af2082 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:20.455605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:20.463579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544bde3-6b7f-4e92-9f2c-d790ef96d9b1 tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81b4918dd4043a0846367084614c083', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60a8e23a1a3147d3a6efd25d1f3653b5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:20.464757 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544bde3-6b7f-4e92-9f2c-d790ef96d9b1 tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:20.465053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544bde3-6b7f-4e92-9f2c-d790ef96d9b1 tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:20.465365 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544bde3-6b7f-4e92-9f2c-d790ef96d9b1 tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:20.466084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544bde3-6b7f-4e92-9f2c-d790ef96d9b1 tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:07:20.466403 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544bde3-6b7f-4e92-9f2c-d790ef96d9b1 tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:20.468777 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544bde3-6b7f-4e92-9f2c-d790ef96d9b1 tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:20.487838 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 982/1947] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:07:20.487838 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544bde3-6b7f-4e92-9f2c-d790ef96d9b1 tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:20.502564 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a544bde3-6b7f-4e92-9f2c-d790ef96d9b1 tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:20.521090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544bde3-6b7f-4e92-9f2c-d790ef96d9b1 tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:20.526409 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018af83f-48f2-4081-b33e-48ef2cfff391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:20.530936 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018af83f-48f2-4081-b33e-48ef2cfff391 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:20.530936 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018af83f-48f2-4081-b33e-48ef2cfff391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:20.530936 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018af83f-48f2-4081-b33e-48ef2cfff391 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:20.530936 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018af83f-48f2-4081-b33e-48ef2cfff391 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:20.530936 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018af83f-48f2-4081-b33e-48ef2cfff391 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:20.536763 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 966/1948] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:20 2022] POST /identity/v3/projects => generated 387 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:20.538118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ddac0-00d9-4f34-b42e-c3003116a5b5 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:20.620559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ddac0-00d9-4f34-b42e-c3003116a5b5 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:20.626667 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de6865-dbe0-4e4a-bd90-ee02fda35885 tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81b4918dd4043a0846367084614c083', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60a8e23a1a3147d3a6efd25d1f3653b5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:20.628022 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de6865-dbe0-4e4a-bd90-ee02fda35885 tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:20.628321 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de6865-dbe0-4e4a-bd90-ee02fda35885 tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:20.628603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de6865-dbe0-4e4a-bd90-ee02fda35885 tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:20.629259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de6865-dbe0-4e4a-bd90-ee02fda35885 tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:20.631401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de6865-dbe0-4e4a-bd90-ee02fda35885 tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:20.678207 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 983/1949] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:20 2022] POST /identity/v3/projects => generated 409 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:20.680014 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018af83f-48f2-4081-b33e-48ef2cfff391 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:20.740748 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 967/1950] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:20 2022] POST /identity/v3/projects => generated 354 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:20.741986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de6865-dbe0-4e4a-bd90-ee02fda35885 tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:20.778686 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018af83f-48f2-4081-b33e-48ef2cfff391 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:20.845506 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de6865-dbe0-4e4a-bd90-ee02fda35885 tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:20.850792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d97b08-9c54-4f98-8de6-8df2a7790e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:20.851557 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d97b08-9c54-4f98-8de6-8df2a7790e56 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:20.851885 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d97b08-9c54-4f98-8de6-8df2a7790e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:20.852136 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d97b08-9c54-4f98-8de6-8df2a7790e56 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:20.852772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d97b08-9c54-4f98-8de6-8df2a7790e56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:20.854755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d97b08-9c54-4f98-8de6-8df2a7790e56 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:20.877245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c275d6-b91f-4333-9355-53e716eeae82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:20.877937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c275d6-b91f-4333-9355-53e716eeae82 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:20.878227 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c275d6-b91f-4333-9355-53e716eeae82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:20.878534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c275d6-b91f-4333-9355-53e716eeae82 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:20.879219 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c275d6-b91f-4333-9355-53e716eeae82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:20.881179 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c275d6-b91f-4333-9355-53e716eeae82 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:20.975571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 968/1951] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:20 2022] POST /identity/v3/users => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:20.976642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d97b08-9c54-4f98-8de6-8df2a7790e56 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:21.051297 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 984/1952] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:20 2022] POST /identity/v3/users => generated 294 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:21.052465 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c275d6-b91f-4333-9355-53e716eeae82 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:21.054398 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-89c275d6-b91f-4333-9355-53e716eeae82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:21.060003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d97b08-9c54-4f98-8de6-8df2a7790e56 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:21.067359 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c275d6-b91f-4333-9355-53e716eeae82 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:21.068221 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430f2a8-8a8d-425f-9e26-576f887d63ad tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81b4918dd4043a0846367084614c083', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60a8e23a1a3147d3a6efd25d1f3653b5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:21.069097 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430f2a8-8a8d-425f-9e26-576f887d63ad tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:21.069583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430f2a8-8a8d-425f-9e26-576f887d63ad tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:21.070186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430f2a8-8a8d-425f-9e26-576f887d63ad tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:21.071029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430f2a8-8a8d-425f-9e26-576f887d63ad tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:07:21.071498 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430f2a8-8a8d-425f-9e26-576f887d63ad tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:21.072447 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913c4b2-a241-4ee2-8f43-25ce029b1cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:21.073099 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913c4b2-a241-4ee2-8f43-25ce029b1cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:21.073416 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913c4b2-a241-4ee2-8f43-25ce029b1cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:21.073692 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913c4b2-a241-4ee2-8f43-25ce029b1cd7 admin admin] PATH_INFO: `/v3/projects/9be7c44c77884a1a87e7f280b2f0c7e7` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:21.074470 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430f2a8-8a8d-425f-9e26-576f887d63ad tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:21.074886 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913c4b2-a241-4ee2-8f43-25ce029b1cd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9be7c44c77884a1a87e7f280b2f0c7e7)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:21.076869 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913c4b2-a241-4ee2-8f43-25ce029b1cd7 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:21.094498 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 969/1953] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:07:21.098886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430f2a8-8a8d-425f-9e26-576f887d63ad tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:21.113970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430f2a8-8a8d-425f-9e26-576f887d63ad tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:21.119320 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932be51-882e-48c7-ae23-b3090a8e95e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:21.119941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932be51-882e-48c7-ae23-b3090a8e95e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:21.120228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932be51-882e-48c7-ae23-b3090a8e95e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:21.120504 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932be51-882e-48c7-ae23-b3090a8e95e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:21.121494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932be51-882e-48c7-ae23-b3090a8e95e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:21.123190 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932be51-882e-48c7-ae23-b3090a8e95e0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:21.139920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 970/1954] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:21.139920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932be51-882e-48c7-ae23-b3090a8e95e0 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:21.150715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932be51-882e-48c7-ae23-b3090a8e95e0 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:21.155629 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd0b82-cc54-4f00-90c7-6e4a11a913a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:21.156282 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd0b82-cc54-4f00-90c7-6e4a11a913a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:21.156571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd0b82-cc54-4f00-90c7-6e4a11a913a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:21.156853 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd0b82-cc54-4f00-90c7-6e4a11a913a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:21.157654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd0b82-cc54-4f00-90c7-6e4a11a913a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:21.159323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd0b82-cc54-4f00-90c7-6e4a11a913a7 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:21.170331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 971/1955] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:21.172479 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd0b82-cc54-4f00-90c7-6e4a11a913a7 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:21.182907 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd0b82-cc54-4f00-90c7-6e4a11a913a7 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:21.187720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe0080-5200-4990-889b-35d129d6cd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:21.188378 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe0080-5200-4990-889b-35d129d6cd80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:21.188663 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe0080-5200-4990-889b-35d129d6cd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:21.188946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe0080-5200-4990-889b-35d129d6cd80 admin admin] PATH_INFO: `/v3/projects/e2386a781f3d44ee92376b6d606de2dd/users/09aa8b184d8f4f52947d7fea43914f0e/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:21.208472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe0080-5200-4990-889b-35d129d6cd80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2386a781f3d44ee92376b6d606de2dd, user_id=09aa8b184d8f4f52947d7fea43914f0e, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:21.211323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe0080-5200-4990-889b-35d129d6cd80 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:21.231147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5913c4b2-a241-4ee2-8f43-25ce029b1cd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9be7c44c77884a1a87e7f280b2f0c7e7', 'request_id': 'req-5913c4b2-a241-4ee2-8f43-25ce029b1cd7'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:21.325585 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 972/1956] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:21 2022] PUT /identity/v3/projects/e2386a781f3d44ee92376b6d606de2dd/users/09aa8b184d8f4f52947d7fea43914f0e/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:21.326408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe0080-5200-4990-889b-35d129d6cd80 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:21.372428 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:21.373688 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5913c4b2-a241-4ee2-8f43-25ce029b1cd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9be7c44c77884a1a87e7f280b2f0c7e7', 'request_id': 'req-5913c4b2-a241-4ee2-8f43-25ce029b1cd7'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:21.385403 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5913c4b2-a241-4ee2-8f43-25ce029b1cd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9be7c44c77884a1a87e7f280b2f0c7e7', 'request_id': 'req-5913c4b2-a241-4ee2-8f43-25ce029b1cd7'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:21.400545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5913c4b2-a241-4ee2-8f43-25ce029b1cd7 admin admin] The token cache is being invalidate because project 9be7c44c77884a1a87e7f280b2f0c7e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:21.400874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5913c4b2-a241-4ee2-8f43-25ce029b1cd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:21.401638 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5913c4b2-a241-4ee2-8f43-25ce029b1cd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:21.404325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 985/1957] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:21 2022] DELETE /identity/v3/projects/9be7c44c77884a1a87e7f280b2f0c7e7 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:21.404917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913c4b2-a241-4ee2-8f43-25ce029b1cd7 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:21.408363 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5913c4b2-a241-4ee2-8f43-25ce029b1cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:21.435462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe0080-5200-4990-889b-35d129d6cd80 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:21.441336 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e500a7b-9911-4cc1-b3f5-b21b3906bb5e tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81b4918dd4043a0846367084614c083', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60a8e23a1a3147d3a6efd25d1f3653b5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:21.442045 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e500a7b-9911-4cc1-b3f5-b21b3906bb5e tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:21.442412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e500a7b-9911-4cc1-b3f5-b21b3906bb5e tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:21.442671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e500a7b-9911-4cc1-b3f5-b21b3906bb5e tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] PATH_INFO: `/v3/projects/f06688a1a0db4af384290ff4a66ae936` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:21.444021 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e500a7b-9911-4cc1-b3f5-b21b3906bb5e tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f06688a1a0db4af384290ff4a66ae936)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:21.446915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e500a7b-9911-4cc1-b3f5-b21b3906bb5e tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:21.491493 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913c4b2-a241-4ee2-8f43-25ce029b1cd7 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:21.555403 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c10873-8ec5-4568-bf31-6cabbc805a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:21.556305 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c10873-8ec5-4568-bf31-6cabbc805a5a admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:21.556570 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c10873-8ec5-4568-bf31-6cabbc805a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:21.556851 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c10873-8ec5-4568-bf31-6cabbc805a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:21.557515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c10873-8ec5-4568-bf31-6cabbc805a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:21.559238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c10873-8ec5-4568-bf31-6cabbc805a5a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:21.573508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 986/1958] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:21.573508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c10873-8ec5-4568-bf31-6cabbc805a5a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:21.578774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c7c10873-8ec5-4568-bf31-6cabbc805a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:21.590849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:21.592124 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c10873-8ec5-4568-bf31-6cabbc805a5a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:21.600687 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-8e500a7b-9911-4cc1-b3f5-b21b3906bb5e tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f06688a1a0db4af384290ff4a66ae936', 'request_id': 'req-8e500a7b-9911-4cc1-b3f5-b21b3906bb5e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:21.656095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61a2df-83c0-487d-9d0d-5745b685c55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:21.656725 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61a2df-83c0-487d-9d0d-5745b685c55b admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:21.657012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61a2df-83c0-487d-9d0d-5745b685c55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:21.657294 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61a2df-83c0-487d-9d0d-5745b685c55b admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:21.657995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61a2df-83c0-487d-9d0d-5745b685c55b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:21.659621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61a2df-83c0-487d-9d0d-5745b685c55b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:21.673111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 987/1959] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:21.673111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e440c02-dfbb-45bf-9553-000af3f3ce4c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:21.673111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e440c02-dfbb-45bf-9553-000af3f3ce4c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:21.673111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e440c02-dfbb-45bf-9553-000af3f3ce4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:21.697063 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4e440c02-dfbb-45bf-9553-000af3f3ce4c None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:21.697063 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4e440c02-dfbb-45bf-9553-000af3f3ce4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:21.709997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 988/1960] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:21.709997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e440c02-dfbb-45bf-9553-000af3f3ce4c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:21.719968 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e440c02-dfbb-45bf-9553-000af3f3ce4c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:21.725537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825ce65-4aa7-4756-96f8-6919fe9241ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:21.726263 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825ce65-4aa7-4756-96f8-6919fe9241ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:21.726559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825ce65-4aa7-4756-96f8-6919fe9241ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:21.727001 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825ce65-4aa7-4756-96f8-6919fe9241ee admin admin] PATH_INFO: `/v3/projects/789a55dc49cb471aa7850a5e250d23ac/users/6b5bb70b69864e2bbb211197082c1bf6/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:21.737961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-8e500a7b-9911-4cc1-b3f5-b21b3906bb5e tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f06688a1a0db4af384290ff4a66ae936', 'request_id': 'req-8e500a7b-9911-4cc1-b3f5-b21b3906bb5e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:21.744409 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825ce65-4aa7-4756-96f8-6919fe9241ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=789a55dc49cb471aa7850a5e250d23ac, user_id=6b5bb70b69864e2bbb211197082c1bf6, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:21.745158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-8e500a7b-9911-4cc1-b3f5-b21b3906bb5e tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f06688a1a0db4af384290ff4a66ae936', 'request_id': 'req-8e500a7b-9911-4cc1-b3f5-b21b3906bb5e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:21.748163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825ce65-4aa7-4756-96f8-6919fe9241ee admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:21.761267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-8e500a7b-9911-4cc1-b3f5-b21b3906bb5e tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] The token cache is being invalidate because project f06688a1a0db4af384290ff4a66ae936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:21.761601 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-8e500a7b-9911-4cc1-b3f5-b21b3906bb5e tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:21.762294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-8e500a7b-9911-4cc1-b3f5-b21b3906bb5e tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:21.763845 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 973/1961] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:21 2022] DELETE /identity/v3/projects/f06688a1a0db4af384290ff4a66ae936 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:21.768251 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e500a7b-9911-4cc1-b3f5-b21b3906bb5e tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:21.769575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8e500a7b-9911-4cc1-b3f5-b21b3906bb5e tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:21.798344 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e500a7b-9911-4cc1-b3f5-b21b3906bb5e tempest-VolumeQuotaClassesTest-1088190052 tempest-VolumeQuotaClassesTest-1088190052-project-admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:21.814715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c28bec-6666-4166-bdce-7721d01e946e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:21.815461 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c28bec-6666-4166-bdce-7721d01e946e admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:21.815720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c28bec-6666-4166-bdce-7721d01e946e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:21.815996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c28bec-6666-4166-bdce-7721d01e946e admin admin] PATH_INFO: `/v3/projects/e2386a781f3d44ee92376b6d606de2dd/users/09aa8b184d8f4f52947d7fea43914f0e/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:21.828329 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c28bec-6666-4166-bdce-7721d01e946e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2386a781f3d44ee92376b6d606de2dd, user_id=09aa8b184d8f4f52947d7fea43914f0e, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:21.838108 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c28bec-6666-4166-bdce-7721d01e946e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:21.853203 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 974/1962] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:21 2022] PUT /identity/v3/projects/e2386a781f3d44ee92376b6d606de2dd/users/09aa8b184d8f4f52947d7fea43914f0e/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:21.854543 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dcffef-ee0c-4f34-8188-08b869b7d870 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:21.854840 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dcffef-ee0c-4f34-8188-08b869b7d870 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:21.855127 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dcffef-ee0c-4f34-8188-08b869b7d870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:21.870058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 989/1963] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:21 2022] PUT /identity/v3/projects/789a55dc49cb471aa7850a5e250d23ac/users/6b5bb70b69864e2bbb211197082c1bf6/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:21.881921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19509170-b1b9-47d3-90d9-cec930f29070 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:21.881921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19509170-b1b9-47d3-90d9-cec930f29070 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:21.881921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19509170-b1b9-47d3-90d9-cec930f29070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:21.893575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-06dcffef-ee0c-4f34-8188-08b869b7d870 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:21.895934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-06dcffef-ee0c-4f34-8188-08b869b7d870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:21.923059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-19509170-b1b9-47d3-90d9-cec930f29070 None None] MFA Rules not processed for user `09aa8b184d8f4f52947d7fea43914f0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:21.924506 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-19509170-b1b9-47d3-90d9-cec930f29070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:22.018274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 975/1964] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:22.020482 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f034d-beb8-4d62-9468-3891bfd9a1e2 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:22.020972 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f034d-beb8-4d62-9468-3891bfd9a1e2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:22.021847 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f034d-beb8-4d62-9468-3891bfd9a1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:22.032939 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 990/1965] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:21 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:22.034441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19509170-b1b9-47d3-90d9-cec930f29070 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:22.045472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19509170-b1b9-47d3-90d9-cec930f29070 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:22.050901 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25908f-f8f8-413a-80c6-262df4949d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:22.051562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25908f-f8f8-413a-80c6-262df4949d6e admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:22.052039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25908f-f8f8-413a-80c6-262df4949d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:22.052832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25908f-f8f8-413a-80c6-262df4949d6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:22.053959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25908f-f8f8-413a-80c6-262df4949d6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:07:22.054300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25908f-f8f8-413a-80c6-262df4949d6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:22.056100 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25908f-f8f8-413a-80c6-262df4949d6e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:22.065322 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c20f034d-beb8-4d62-9468-3891bfd9a1e2 None None] MFA Rules not processed for user `6b5bb70b69864e2bbb211197082c1bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:22.066600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c20f034d-beb8-4d62-9468-3891bfd9a1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:22.069282 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 991/1966] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:07:22.070178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25908f-f8f8-413a-80c6-262df4949d6e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:22.076390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fb25908f-f8f8-413a-80c6-262df4949d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:22.147100 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25908f-f8f8-413a-80c6-262df4949d6e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:22.151245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cdf5a-04f3-45fe-bf3b-ff5c6b4665d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:22.151944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cdf5a-04f3-45fe-bf3b-ff5c6b4665d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:22.152300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cdf5a-04f3-45fe-bf3b-ff5c6b4665d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:22.152648 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cdf5a-04f3-45fe-bf3b-ff5c6b4665d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:22.153970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 976/1967] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:22.155114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f034d-beb8-4d62-9468-3891bfd9a1e2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:22.166068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cdf5a-04f3-45fe-bf3b-ff5c6b4665d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:22.167604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cdf5a-04f3-45fe-bf3b-ff5c6b4665d1 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:22.172675 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f034d-beb8-4d62-9468-3891bfd9a1e2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:22.177891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf70d98-a900-48dd-a5be-c204e9514fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:22.178527 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf70d98-a900-48dd-a5be-c204e9514fce admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:22.178895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf70d98-a900-48dd-a5be-c204e9514fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:22.179360 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf70d98-a900-48dd-a5be-c204e9514fce admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:22.180076 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf70d98-a900-48dd-a5be-c204e9514fce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:22.183208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf70d98-a900-48dd-a5be-c204e9514fce admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:22.187298 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 992/1968] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:22 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:22.190866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cdf5a-04f3-45fe-bf3b-ff5c6b4665d1 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:22.192365 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-304cdf5a-04f3-45fe-bf3b-ff5c6b4665d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:22.206766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cdf5a-04f3-45fe-bf3b-ff5c6b4665d1 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:22.213917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd8062-8f84-4f6f-8e7c-4d01a3d31847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:22.215240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd8062-8f84-4f6f-8e7c-4d01a3d31847 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:22.215598 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd8062-8f84-4f6f-8e7c-4d01a3d31847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:22.216188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd8062-8f84-4f6f-8e7c-4d01a3d31847 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:22.217381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd8062-8f84-4f6f-8e7c-4d01a3d31847 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:22.220044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd8062-8f84-4f6f-8e7c-4d01a3d31847 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:22.342346 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 977/1969] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:22 2022] POST /identity/v3/projects => generated 373 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:22.347054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf70d98-a900-48dd-a5be-c204e9514fce admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:22.417752 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf70d98-a900-48dd-a5be-c204e9514fce admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:22.420757 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 993/1970] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:22 2022] POST /identity/v3/projects => generated 387 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:22.423302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39bf42d-8b45-44bb-892b-0a1cc618180a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:22.423932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39bf42d-8b45-44bb-892b-0a1cc618180a admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:22.424506 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39bf42d-8b45-44bb-892b-0a1cc618180a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:22.424805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39bf42d-8b45-44bb-892b-0a1cc618180a admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:22.425469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39bf42d-8b45-44bb-892b-0a1cc618180a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:22.425616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd8062-8f84-4f6f-8e7c-4d01a3d31847 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:22.427442 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39bf42d-8b45-44bb-892b-0a1cc618180a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:22.496033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd8062-8f84-4f6f-8e7c-4d01a3d31847 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:22.500868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb16660-243f-44e2-8a84-ab46f7dc24a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:22.501540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb16660-243f-44e2-8a84-ab46f7dc24a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:22.501827 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb16660-243f-44e2-8a84-ab46f7dc24a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:22.502138 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb16660-243f-44e2-8a84-ab46f7dc24a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:22.502787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb16660-243f-44e2-8a84-ab46f7dc24a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:22.504726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb16660-243f-44e2-8a84-ab46f7dc24a3 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:22.630744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 978/1971] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:22 2022] POST /identity/v3/users => generated 287 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:07:22.641744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39bf42d-8b45-44bb-892b-0a1cc618180a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:22.659229 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39bf42d-8b45-44bb-892b-0a1cc618180a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:22.665076 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89f4e0-5180-40ce-b9b0-6ee69cafdf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:22.665699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89f4e0-5180-40ce-b9b0-6ee69cafdf22 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:22.666018 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89f4e0-5180-40ce-b9b0-6ee69cafdf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:22.666301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89f4e0-5180-40ce-b9b0-6ee69cafdf22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:22.666959 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89f4e0-5180-40ce-b9b0-6ee69cafdf22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:22.668811 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89f4e0-5180-40ce-b9b0-6ee69cafdf22 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:22.671090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 994/1972] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:22 2022] POST /identity/v3/users => generated 300 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:07:22.678912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb16660-243f-44e2-8a84-ab46f7dc24a3 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:22.686227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 979/1973] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:07:22.687428 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89f4e0-5180-40ce-b9b0-6ee69cafdf22 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:22.690244 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb16660-243f-44e2-8a84-ab46f7dc24a3 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:22.698324 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9fbca-8d08-41c0-b2f0-340784e900f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:22.699060 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9fbca-8d08-41c0-b2f0-340784e900f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:22.699415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9fbca-8d08-41c0-b2f0-340784e900f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:22.699521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89f4e0-5180-40ce-b9b0-6ee69cafdf22 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:22.699836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9fbca-8d08-41c0-b2f0-340784e900f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:22.700615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9fbca-8d08-41c0-b2f0-340784e900f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:22.702406 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9fbca-8d08-41c0-b2f0-340784e900f9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:22.706357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61d688-3169-4e87-9d89-1e505f08c697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:22.706990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61d688-3169-4e87-9d89-1e505f08c697 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:22.707277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61d688-3169-4e87-9d89-1e505f08c697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:22.707573 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61d688-3169-4e87-9d89-1e505f08c697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:22.708364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf61d688-3169-4e87-9d89-1e505f08c697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:22.711543 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf61d688-3169-4e87-9d89-1e505f08c697 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:22.718013 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 995/1974] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:07:22.721033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9fbca-8d08-41c0-b2f0-340784e900f9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:22.725100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 980/1975] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:22.729379 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61d688-3169-4e87-9d89-1e505f08c697 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:22.732859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9fbca-8d08-41c0-b2f0-340784e900f9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:22.741333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03327d-4377-4417-bea8-bcb28f8d4abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:22.742101 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03327d-4377-4417-bea8-bcb28f8d4abb admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:22.747926 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03327d-4377-4417-bea8-bcb28f8d4abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:22.747926 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03327d-4377-4417-bea8-bcb28f8d4abb admin admin] PATH_INFO: `/v3/projects/29731a9b73fe4c968139d735e3f414e4/users/27c87b2f3f1840b49e60be0e98f0e7ab/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:22.749157 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61d688-3169-4e87-9d89-1e505f08c697 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:22.759871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788d64c-dac6-4b80-a018-ff7471aa3d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:22.761789 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788d64c-dac6-4b80-a018-ff7471aa3d80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:22.761948 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03327d-4377-4417-bea8-bcb28f8d4abb admin admin] RBAC: Authorizing `identity:create_grant(project_id=29731a9b73fe4c968139d735e3f414e4, user_id=27c87b2f3f1840b49e60be0e98f0e7ab, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:22.762401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788d64c-dac6-4b80-a018-ff7471aa3d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:22.763685 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788d64c-dac6-4b80-a018-ff7471aa3d80 admin admin] PATH_INFO: `/v3/projects/32fa57f1c5f84cc3b19af11b61e85e58/users/c01685093f1e4457a53a3deaf309d14d/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:22.764998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03327d-4377-4417-bea8-bcb28f8d4abb admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:22.787325 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788d64c-dac6-4b80-a018-ff7471aa3d80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32fa57f1c5f84cc3b19af11b61e85e58, user_id=c01685093f1e4457a53a3deaf309d14d, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:22.790773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788d64c-dac6-4b80-a018-ff7471aa3d80 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:22.900915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 996/1976] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:22 2022] PUT /identity/v3/projects/29731a9b73fe4c968139d735e3f414e4/users/27c87b2f3f1840b49e60be0e98f0e7ab/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:07:22.904428 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03327d-4377-4417-bea8-bcb28f8d4abb admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:22.922002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:22.930180 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 981/1977] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:22 2022] PUT /identity/v3/projects/32fa57f1c5f84cc3b19af11b61e85e58/users/c01685093f1e4457a53a3deaf309d14d/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:07:22.933221 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788d64c-dac6-4b80-a018-ff7471aa3d80 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:23.012199 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03327d-4377-4417-bea8-bcb28f8d4abb admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:23.033771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788d64c-dac6-4b80-a018-ff7471aa3d80 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:23.041476 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f27bf-1512-464a-a023-873f827ccbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:23.042141 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f27bf-1512-464a-a023-873f827ccbf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:23.042432 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f27bf-1512-464a-a023-873f827ccbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:23.042711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f27bf-1512-464a-a023-873f827ccbf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:23.043390 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f27bf-1512-464a-a023-873f827ccbf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:23.045267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f27bf-1512-464a-a023-873f827ccbf0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:23.050770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:23.064834 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 982/1978] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:23.066398 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f610f4-2b9e-4dd9-a00f-7f9ae59c31c7 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:23.066693 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f610f4-2b9e-4dd9-a00f-7f9ae59c31c7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:23.067044 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f610f4-2b9e-4dd9-a00f-7f9ae59c31c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:23.109406 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f25a21-8809-4a67-9e57-702ddac6eed0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:23.110430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f25a21-8809-4a67-9e57-702ddac6eed0 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:23.111517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f25a21-8809-4a67-9e57-702ddac6eed0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:23.111872 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f25a21-8809-4a67-9e57-702ddac6eed0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:23.114887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b4f25a21-8809-4a67-9e57-702ddac6eed0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:23.120771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a1f610f4-2b9e-4dd9-a00f-7f9ae59c31c7 None None] MFA Rules not processed for user `c01685093f1e4457a53a3deaf309d14d`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:23.123063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a1f610f4-2b9e-4dd9-a00f-7f9ae59c31c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:23.213932 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f25a21-8809-4a67-9e57-702ddac6eed0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:23.216010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f25a21-8809-4a67-9e57-702ddac6eed0 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:23.226769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 983/1979] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:23.226769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f610f4-2b9e-4dd9-a00f-7f9ae59c31c7 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:23.228082 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 997/1980] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:22 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:23.229249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f25a21-8809-4a67-9e57-702ddac6eed0 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:23.230682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b4f25a21-8809-4a67-9e57-702ddac6eed0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:23.249841 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f25a21-8809-4a67-9e57-702ddac6eed0 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:23.261244 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167ffeb-03ad-46dc-9a25-2ea60d8457af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:23.261754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f610f4-2b9e-4dd9-a00f-7f9ae59c31c7 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:23.263941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167ffeb-03ad-46dc-9a25-2ea60d8457af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:23.264449 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167ffeb-03ad-46dc-9a25-2ea60d8457af admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:23.267100 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167ffeb-03ad-46dc-9a25-2ea60d8457af admin admin] PATH_INFO: `/v3/users/730a9fe071824d4799e46051db0797de` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:23.269103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b76e18-5101-4728-88df-c155149d880e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:23.269795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b76e18-5101-4728-88df-c155149d880e admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:23.270143 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b76e18-5101-4728-88df-c155149d880e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:23.270408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b76e18-5101-4728-88df-c155149d880e admin admin] PATH_INFO: `/v3/projects/29731a9b73fe4c968139d735e3f414e4/users/27c87b2f3f1840b49e60be0e98f0e7ab/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:23.270634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167ffeb-03ad-46dc-9a25-2ea60d8457af admin admin] RBAC: Authorizing `identity:delete_user(user_id=730a9fe071824d4799e46051db0797de)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:23.272749 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b76e18-5101-4728-88df-c155149d880e admin admin] RBAC: Authorizing `identity:create_grant(project_id=29731a9b73fe4c968139d735e3f414e4, user_id=27c87b2f3f1840b49e60be0e98f0e7ab, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:23.274230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167ffeb-03ad-46dc-9a25-2ea60d8457af admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:23.276021 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b76e18-5101-4728-88df-c155149d880e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:23.288672 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 984/1981] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:23 2022] PUT /identity/v3/projects/29731a9b73fe4c968139d735e3f414e4/users/27c87b2f3f1840b49e60be0e98f0e7ab/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:23.298217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:23.299045 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b76e18-5101-4728-88df-c155149d880e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:23.301060 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f0b76e18-5101-4728-88df-c155149d880e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:23.376198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b76e18-5101-4728-88df-c155149d880e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:23.380022 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfede623-2d67-4d82-9592-0c06f4f13e44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:23.380811 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfede623-2d67-4d82-9592-0c06f4f13e44 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:23.381098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfede623-2d67-4d82-9592-0c06f4f13e44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:23.381374 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfede623-2d67-4d82-9592-0c06f4f13e44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:23.462959 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfede623-2d67-4d82-9592-0c06f4f13e44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:23.465940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfede623-2d67-4d82-9592-0c06f4f13e44 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:23.477056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 985/1982] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:23.479550 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379db281-6e3f-4d0d-a403-7b9d77f24375 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:23.480003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379db281-6e3f-4d0d-a403-7b9d77f24375 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:23.480391 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379db281-6e3f-4d0d-a403-7b9d77f24375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:23.511167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b167ffeb-03ad-46dc-9a25-2ea60d8457af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '730a9fe071824d4799e46051db0797de'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:23.533391 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b167ffeb-03ad-46dc-9a25-2ea60d8457af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '730a9fe071824d4799e46051db0797de'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:23.536331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-379db281-6e3f-4d0d-a403-7b9d77f24375 None None] MFA Rules not processed for user `27c87b2f3f1840b49e60be0e98f0e7ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:23.537920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-379db281-6e3f-4d0d-a403-7b9d77f24375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:23.543163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b167ffeb-03ad-46dc-9a25-2ea60d8457af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '730a9fe071824d4799e46051db0797de'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:23.595929 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:23.629461 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 986/1983] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:23 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:23.634227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379db281-6e3f-4d0d-a403-7b9d77f24375 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:23.657104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b167ffeb-03ad-46dc-9a25-2ea60d8457af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '730a9fe071824d4799e46051db0797de'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:23.657826 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b167ffeb-03ad-46dc-9a25-2ea60d8457af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '730a9fe071824d4799e46051db0797de'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:23.664510 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 998/1984] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:23 2022] DELETE /identity/v3/users/730a9fe071824d4799e46051db0797de => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:23.668163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167ffeb-03ad-46dc-9a25-2ea60d8457af admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:23.669164 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b167ffeb-03ad-46dc-9a25-2ea60d8457af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:23.711076 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379db281-6e3f-4d0d-a403-7b9d77f24375 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:23.748853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167ffeb-03ad-46dc-9a25-2ea60d8457af admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:23.759120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667dc05-7261-4ebf-a22d-65bd099e91d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:23.760317 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667dc05-7261-4ebf-a22d-65bd099e91d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:23.760806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667dc05-7261-4ebf-a22d-65bd099e91d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:23.761238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667dc05-7261-4ebf-a22d-65bd099e91d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:23.763637 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9667dc05-7261-4ebf-a22d-65bd099e91d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:23.770060 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:23.789144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca01ca8-0207-48ac-80fd-22db924f8bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:23.789961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca01ca8-0207-48ac-80fd-22db924f8bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:23.790150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca01ca8-0207-48ac-80fd-22db924f8bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:23.790461 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca01ca8-0207-48ac-80fd-22db924f8bb5 admin admin] PATH_INFO: `/v3/users/c01685093f1e4457a53a3deaf309d14d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:23.791673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca01ca8-0207-48ac-80fd-22db924f8bb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c01685093f1e4457a53a3deaf309d14d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:23.793720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca01ca8-0207-48ac-80fd-22db924f8bb5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:23.864461 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667dc05-7261-4ebf-a22d-65bd099e91d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:23.866306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667dc05-7261-4ebf-a22d-65bd099e91d5 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:23.878540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 999/1985] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:23 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:23.879895 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d74f25-21e7-48bc-83f0-faca080633dd None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:23.880274 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d74f25-21e7-48bc-83f0-faca080633dd None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:23.880629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d74f25-21e7-48bc-83f0-faca080633dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:23.928263 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f2d74f25-21e7-48bc-83f0-faca080633dd None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:23.932945 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f2d74f25-21e7-48bc-83f0-faca080633dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:23.957081 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1000/1986] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:23.965142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d74f25-21e7-48bc-83f0-faca080633dd None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:23.967061 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f2d74f25-21e7-48bc-83f0-faca080633dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:24.056377 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d74f25-21e7-48bc-83f0-faca080633dd None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:24.060583 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ce96f-d986-4b43-8a69-2523f78b27cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:24.061810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ce96f-d986-4b43-8a69-2523f78b27cd service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:24.061810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ce96f-d986-4b43-8a69-2523f78b27cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:24.061810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ce96f-d986-4b43-8a69-2523f78b27cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:24.074388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ce96f-d986-4b43-8a69-2523f78b27cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:24.080649 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ce96f-d986-4b43-8a69-2523f78b27cd service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:24.087429 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1001/1987] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:24.088538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ce96f-d986-4b43-8a69-2523f78b27cd service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:24.090059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c29ce96f-d986-4b43-8a69-2523f78b27cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:24.107947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ce96f-d986-4b43-8a69-2523f78b27cd service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:24.114956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f21723-1ba7-451a-99c0-70d5d25abf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:24.115618 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f21723-1ba7-451a-99c0-70d5d25abf2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:24.115892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f21723-1ba7-451a-99c0-70d5d25abf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:24.116174 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f21723-1ba7-451a-99c0-70d5d25abf2b admin admin] PATH_INFO: `/v3/users/6b5bb70b69864e2bbb211197082c1bf6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:24.117511 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f21723-1ba7-451a-99c0-70d5d25abf2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b5bb70b69864e2bbb211197082c1bf6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:24.120149 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f21723-1ba7-451a-99c0-70d5d25abf2b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:24.171095 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0ca01ca8-0207-48ac-80fd-22db924f8bb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c01685093f1e4457a53a3deaf309d14d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:24.203063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0ca01ca8-0207-48ac-80fd-22db924f8bb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c01685093f1e4457a53a3deaf309d14d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:24.206669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:24.217790 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0ca01ca8-0207-48ac-80fd-22db924f8bb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c01685093f1e4457a53a3deaf309d14d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:24.343114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0ca01ca8-0207-48ac-80fd-22db924f8bb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c01685093f1e4457a53a3deaf309d14d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:24.343114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0ca01ca8-0207-48ac-80fd-22db924f8bb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c01685093f1e4457a53a3deaf309d14d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:24.352707 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 987/1988] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:23 2022] DELETE /identity/v3/users/c01685093f1e4457a53a3deaf309d14d => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:24.358413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32666524-4e40-4590-a2aa-94e74c0df794 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:24.359990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32666524-4e40-4590-a2aa-94e74c0df794 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:24.360429 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32666524-4e40-4590-a2aa-94e74c0df794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:24.382985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-32666524-4e40-4590-a2aa-94e74c0df794 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:24.384216 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-32666524-4e40-4590-a2aa-94e74c0df794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:24.455237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 988/1989] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:24.470575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32666524-4e40-4590-a2aa-94e74c0df794 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:24.472083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-32666524-4e40-4590-a2aa-94e74c0df794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:24.499295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b5f21723-1ba7-451a-99c0-70d5d25abf2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b5bb70b69864e2bbb211197082c1bf6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:24.507351 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:24.540506 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b5f21723-1ba7-451a-99c0-70d5d25abf2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b5bb70b69864e2bbb211197082c1bf6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:24.549733 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b5f21723-1ba7-451a-99c0-70d5d25abf2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b5bb70b69864e2bbb211197082c1bf6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:24.585673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32666524-4e40-4590-a2aa-94e74c0df794 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:24.596643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3258d-deb3-4020-8050-1b91af135601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:24.599109 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3258d-deb3-4020-8050-1b91af135601 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:24.600132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3258d-deb3-4020-8050-1b91af135601 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:24.613891 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3258d-deb3-4020-8050-1b91af135601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:24.623732 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b5f21723-1ba7-451a-99c0-70d5d25abf2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b5bb70b69864e2bbb211197082c1bf6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:24.624638 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b5f21723-1ba7-451a-99c0-70d5d25abf2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b5bb70b69864e2bbb211197082c1bf6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:24.628465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd3258d-deb3-4020-8050-1b91af135601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:24.635966 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd3258d-deb3-4020-8050-1b91af135601 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:24.636360 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1002/1990] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:24 2022] DELETE /identity/v3/users/6b5bb70b69864e2bbb211197082c1bf6 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:24.638888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e5345-cb83-4244-af83-eb4f7bc19762 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:24.639441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e5345-cb83-4244-af83-eb4f7bc19762 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:24.640103 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e5345-cb83-4244-af83-eb4f7bc19762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:24.663518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 989/1991] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:07:24.666502 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-586e5345-cb83-4244-af83-eb4f7bc19762 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:24.668638 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-586e5345-cb83-4244-af83-eb4f7bc19762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:24.705031 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:24.776080 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1003/1992] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:24.792715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3258d-deb3-4020-8050-1b91af135601 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:24.795297 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3cd3258d-deb3-4020-8050-1b91af135601 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:24.808975 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3258d-deb3-4020-8050-1b91af135601 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:24.820451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:24.873666 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879cd93-8a78-463b-b139-c4dc29666599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:24.875699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879cd93-8a78-463b-b139-c4dc29666599 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:24.878476 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879cd93-8a78-463b-b139-c4dc29666599 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:24.878808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879cd93-8a78-463b-b139-c4dc29666599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:24.892459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e5345-cb83-4244-af83-eb4f7bc19762 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:24.898245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5879cd93-8a78-463b-b139-c4dc29666599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:24.898436 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-586e5345-cb83-4244-af83-eb4f7bc19762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:24.901109 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5879cd93-8a78-463b-b139-c4dc29666599 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:24.919974 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 990/1993] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:24.932418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e5345-cb83-4244-af83-eb4f7bc19762 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:24.941964 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf175004-e7e1-4c12-94ec-09a26ac0a747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:24.942832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf175004-e7e1-4c12-94ec-09a26ac0a747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:24.943182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf175004-e7e1-4c12-94ec-09a26ac0a747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:24.943534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf175004-e7e1-4c12-94ec-09a26ac0a747 admin admin] PATH_INFO: `/v3/projects/9adc0a5801cf49d8a7ce8d1983743a4c` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:24.945400 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf175004-e7e1-4c12-94ec-09a26ac0a747 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9adc0a5801cf49d8a7ce8d1983743a4c)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:24.947885 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf175004-e7e1-4c12-94ec-09a26ac0a747 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:25.120215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cf175004-e7e1-4c12-94ec-09a26ac0a747 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9adc0a5801cf49d8a7ce8d1983743a4c', 'request_id': 'req-cf175004-e7e1-4c12-94ec-09a26ac0a747'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:25.206586 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cf175004-e7e1-4c12-94ec-09a26ac0a747 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9adc0a5801cf49d8a7ce8d1983743a4c', 'request_id': 'req-cf175004-e7e1-4c12-94ec-09a26ac0a747'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:25.217566 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cf175004-e7e1-4c12-94ec-09a26ac0a747 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9adc0a5801cf49d8a7ce8d1983743a4c', 'request_id': 'req-cf175004-e7e1-4c12-94ec-09a26ac0a747'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:25.238546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cf175004-e7e1-4c12-94ec-09a26ac0a747 admin admin] The token cache is being invalidate because project 9adc0a5801cf49d8a7ce8d1983743a4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:25.238987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cf175004-e7e1-4c12-94ec-09a26ac0a747 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:25.239785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cf175004-e7e1-4c12-94ec-09a26ac0a747 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:25.241012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:25.246091 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879cd93-8a78-463b-b139-c4dc29666599 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:25.254799 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1004/1994] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:24 2022] DELETE /identity/v3/projects/9adc0a5801cf49d8a7ce8d1983743a4c => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:25.257581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879cd93-8a78-463b-b139-c4dc29666599 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:25.297127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:25.358130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf175004-e7e1-4c12-94ec-09a26ac0a747 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:25.360333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cf175004-e7e1-4c12-94ec-09a26ac0a747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:25.416451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a102c4a-f040-47f0-8f42-bc3acbe11112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:25.417806 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a102c4a-f040-47f0-8f42-bc3acbe11112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:25.422511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a102c4a-f040-47f0-8f42-bc3acbe11112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:25.424408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a102c4a-f040-47f0-8f42-bc3acbe11112 admin admin] PATH_INFO: `/v3/users/a81b4918dd4043a0846367084614c083` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:25.433757 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a102c4a-f040-47f0-8f42-bc3acbe11112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a81b4918dd4043a0846367084614c083)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:25.442501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a102c4a-f040-47f0-8f42-bc3acbe11112 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:25.510676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf175004-e7e1-4c12-94ec-09a26ac0a747 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:25.520608 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7d39a-2946-4495-92c7-6b8c87a38107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:25.521280 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7d39a-2946-4495-92c7-6b8c87a38107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:25.521569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7d39a-2946-4495-92c7-6b8c87a38107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:25.522155 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7d39a-2946-4495-92c7-6b8c87a38107 admin admin] PATH_INFO: `/v3/projects/32fa57f1c5f84cc3b19af11b61e85e58` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:25.523374 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7d39a-2946-4495-92c7-6b8c87a38107 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32fa57f1c5f84cc3b19af11b61e85e58)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:25.526548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7d39a-2946-4495-92c7-6b8c87a38107 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:25.737636 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-45c7d39a-2946-4495-92c7-6b8c87a38107 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32fa57f1c5f84cc3b19af11b61e85e58', 'request_id': 'req-45c7d39a-2946-4495-92c7-6b8c87a38107'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:25.742576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2a102c4a-f040-47f0-8f42-bc3acbe11112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a81b4918dd4043a0846367084614c083'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:25.779888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2a102c4a-f040-47f0-8f42-bc3acbe11112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a81b4918dd4043a0846367084614c083'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:25.796317 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2a102c4a-f040-47f0-8f42-bc3acbe11112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a81b4918dd4043a0846367084614c083'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:25.812398 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-45c7d39a-2946-4495-92c7-6b8c87a38107 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32fa57f1c5f84cc3b19af11b61e85e58', 'request_id': 'req-45c7d39a-2946-4495-92c7-6b8c87a38107'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:25.828825 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-45c7d39a-2946-4495-92c7-6b8c87a38107 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32fa57f1c5f84cc3b19af11b61e85e58', 'request_id': 'req-45c7d39a-2946-4495-92c7-6b8c87a38107'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:25.854461 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-45c7d39a-2946-4495-92c7-6b8c87a38107 admin admin] The token cache is being invalidate because project 32fa57f1c5f84cc3b19af11b61e85e58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:25.854951 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-45c7d39a-2946-4495-92c7-6b8c87a38107 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:25.855749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-45c7d39a-2946-4495-92c7-6b8c87a38107 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:25.858132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1005/1995] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:25 2022] DELETE /identity/v3/projects/32fa57f1c5f84cc3b19af11b61e85e58 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:25.859176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7d39a-2946-4495-92c7-6b8c87a38107 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:25.860715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-45c7d39a-2946-4495-92c7-6b8c87a38107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:25.939517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7d39a-2946-4495-92c7-6b8c87a38107 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:25.950644 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3d59c-1e40-495a-a849-327a87e01e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:25.951313 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3d59c-1e40-495a-a849-327a87e01e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:25.955543 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3d59c-1e40-495a-a849-327a87e01e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:25.955543 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3d59c-1e40-495a-a849-327a87e01e06 admin admin] PATH_INFO: `/v3/projects/789a55dc49cb471aa7850a5e250d23ac` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:25.955543 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3d59c-1e40-495a-a849-327a87e01e06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=789a55dc49cb471aa7850a5e250d23ac)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:25.955808 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3d59c-1e40-495a-a849-327a87e01e06 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:25.988303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:25.988303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2a102c4a-f040-47f0-8f42-bc3acbe11112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a81b4918dd4043a0846367084614c083'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:25.989025 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2a102c4a-f040-47f0-8f42-bc3acbe11112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a81b4918dd4043a0846367084614c083'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:26.002107 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 991/1996] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:25 2022] DELETE /identity/v3/users/a81b4918dd4043a0846367084614c083 => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:26.003110 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055848e0-4a93-4bbc-b780-f5542b80e41b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:26.003455 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055848e0-4a93-4bbc-b780-f5542b80e41b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:26.003753 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055848e0-4a93-4bbc-b780-f5542b80e41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:26.035212 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-055848e0-4a93-4bbc-b780-f5542b80e41b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:26.036585 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-055848e0-4a93-4bbc-b780-f5542b80e41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:26.147200 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 992/1997] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:26.156560 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444241e0-9199-49ba-886c-d275b6825bd9 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:26.156862 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444241e0-9199-49ba-886c-d275b6825bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:26.157146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444241e0-9199-49ba-886c-d275b6825bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:26.177805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-444241e0-9199-49ba-886c-d275b6825bd9 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:26.178596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7ac3d59c-1e40-495a-a849-327a87e01e06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '789a55dc49cb471aa7850a5e250d23ac', 'request_id': 'req-7ac3d59c-1e40-495a-a849-327a87e01e06'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:26.179101 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-444241e0-9199-49ba-886c-d275b6825bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:26.246361 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 993/1998] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:26.250648 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444241e0-9199-49ba-886c-d275b6825bd9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:26.260225 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444241e0-9199-49ba-886c-d275b6825bd9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:26.267218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2da329-7b0a-434e-b818-1b7105832b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:26.267644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2da329-7b0a-434e-b818-1b7105832b83 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:26.267932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2da329-7b0a-434e-b818-1b7105832b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:26.268213 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2da329-7b0a-434e-b818-1b7105832b83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:26.268885 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2da329-7b0a-434e-b818-1b7105832b83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:07:26.269198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2da329-7b0a-434e-b818-1b7105832b83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:26.270923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2da329-7b0a-434e-b818-1b7105832b83 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:26.290420 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 994/1999] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:07:26.293612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2da329-7b0a-434e-b818-1b7105832b83 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:26.304633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2da329-7b0a-434e-b818-1b7105832b83 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:26.309030 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7ac3d59c-1e40-495a-a849-327a87e01e06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '789a55dc49cb471aa7850a5e250d23ac', 'request_id': 'req-7ac3d59c-1e40-495a-a849-327a87e01e06'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:26.310184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42dc363-0410-4138-b6de-e689f5fc51f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:26.313102 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42dc363-0410-4138-b6de-e689f5fc51f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:26.313102 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42dc363-0410-4138-b6de-e689f5fc51f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:26.313102 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42dc363-0410-4138-b6de-e689f5fc51f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:26.313102 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42dc363-0410-4138-b6de-e689f5fc51f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:26.314514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42dc363-0410-4138-b6de-e689f5fc51f6 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:26.317133 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7ac3d59c-1e40-495a-a849-327a87e01e06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '789a55dc49cb471aa7850a5e250d23ac', 'request_id': 'req-7ac3d59c-1e40-495a-a849-327a87e01e06'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:26.335268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7ac3d59c-1e40-495a-a849-327a87e01e06 admin admin] The token cache is being invalidate because project 789a55dc49cb471aa7850a5e250d23ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:26.335717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7ac3d59c-1e40-495a-a849-327a87e01e06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:26.336715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7ac3d59c-1e40-495a-a849-327a87e01e06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:26.338810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1006/2000] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:25 2022] DELETE /identity/v3/projects/789a55dc49cb471aa7850a5e250d23ac => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:26.342763 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3d59c-1e40-495a-a849-327a87e01e06 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:26.344279 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7ac3d59c-1e40-495a-a849-327a87e01e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:26.358023 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3d59c-1e40-495a-a849-327a87e01e06 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:26.366335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33dd39-0960-4d07-8e70-19e7997f8671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:26.367395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33dd39-0960-4d07-8e70-19e7997f8671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:26.370550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33dd39-0960-4d07-8e70-19e7997f8671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:26.370550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33dd39-0960-4d07-8e70-19e7997f8671 admin admin] PATH_INFO: `/v3/users/27c87b2f3f1840b49e60be0e98f0e7ab` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:26.370550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c33dd39-0960-4d07-8e70-19e7997f8671 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27c87b2f3f1840b49e60be0e98f0e7ab)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:26.373266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c33dd39-0960-4d07-8e70-19e7997f8671 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:26.429642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 995/2001] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:26 2022] POST /identity/v3/projects => generated 377 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:26.434176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42dc363-0410-4138-b6de-e689f5fc51f6 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:26.436120 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d42dc363-0410-4138-b6de-e689f5fc51f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:26.504025 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:26.521260 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42dc363-0410-4138-b6de-e689f5fc51f6 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:26.526426 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd08c67-129d-43f4-be03-de4f5d55efe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:26.527046 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd08c67-129d-43f4-be03-de4f5d55efe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:26.527401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd08c67-129d-43f4-be03-de4f5d55efe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:26.527677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd08c67-129d-43f4-be03-de4f5d55efe5 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:26.528501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd08c67-129d-43f4-be03-de4f5d55efe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:26.530438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd08c67-129d-43f4-be03-de4f5d55efe5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:26.607057 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0c33dd39-0960-4d07-8e70-19e7997f8671 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27c87b2f3f1840b49e60be0e98f0e7ab'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:26.622609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0c33dd39-0960-4d07-8e70-19e7997f8671 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27c87b2f3f1840b49e60be0e98f0e7ab'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:26.635259 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0c33dd39-0960-4d07-8e70-19e7997f8671 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27c87b2f3f1840b49e60be0e98f0e7ab'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:26.667536 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 996/2002] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:26 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:07:26.671057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd08c67-129d-43f4-be03-de4f5d55efe5 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:26.671057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2cd08c67-129d-43f4-be03-de4f5d55efe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:26.685947 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd08c67-129d-43f4-be03-de4f5d55efe5 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:26.696975 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041c73c-7c5e-4576-82a9-2f6c9b1239da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:26.698044 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041c73c-7c5e-4576-82a9-2f6c9b1239da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:26.698469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041c73c-7c5e-4576-82a9-2f6c9b1239da admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:26.699146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041c73c-7c5e-4576-82a9-2f6c9b1239da admin admin] PATH_INFO: `/v3/projects/60a8e23a1a3147d3a6efd25d1f3653b5` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:26.700545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041c73c-7c5e-4576-82a9-2f6c9b1239da admin admin] RBAC: Authorizing `identity:delete_project(project_id=60a8e23a1a3147d3a6efd25d1f3653b5)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:26.704591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041c73c-7c5e-4576-82a9-2f6c9b1239da admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:26.748243 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0c33dd39-0960-4d07-8e70-19e7997f8671 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27c87b2f3f1840b49e60be0e98f0e7ab'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:26.749003 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0c33dd39-0960-4d07-8e70-19e7997f8671 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27c87b2f3f1840b49e60be0e98f0e7ab'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:26.756744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1007/2003] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:26 2022] DELETE /identity/v3/users/27c87b2f3f1840b49e60be0e98f0e7ab => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:26.757534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33dd39-0960-4d07-8e70-19e7997f8671 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:26.759110 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0c33dd39-0960-4d07-8e70-19e7997f8671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:26.879175 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33dd39-0960-4d07-8e70-19e7997f8671 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:26.887088 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0041c73c-7c5e-4576-82a9-2f6c9b1239da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60a8e23a1a3147d3a6efd25d1f3653b5', 'request_id': 'req-0041c73c-7c5e-4576-82a9-2f6c9b1239da'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:26.961225 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7b212-adfa-4869-bd11-22efdcd921f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:26.962310 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7b212-adfa-4869-bd11-22efdcd921f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:26.964073 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7b212-adfa-4869-bd11-22efdcd921f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:26.964463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7b212-adfa-4869-bd11-22efdcd921f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:26.965298 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b7b212-adfa-4869-bd11-22efdcd921f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:26.969090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b7b212-adfa-4869-bd11-22efdcd921f9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:26.978040 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0041c73c-7c5e-4576-82a9-2f6c9b1239da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60a8e23a1a3147d3a6efd25d1f3653b5', 'request_id': 'req-0041c73c-7c5e-4576-82a9-2f6c9b1239da'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:26.978393 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:27.006621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0041c73c-7c5e-4576-82a9-2f6c9b1239da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60a8e23a1a3147d3a6efd25d1f3653b5', 'request_id': 'req-0041c73c-7c5e-4576-82a9-2f6c9b1239da'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:27.006952 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1008/2004] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:27.006952 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7b212-adfa-4869-bd11-22efdcd921f9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:27.007158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0041c73c-7c5e-4576-82a9-2f6c9b1239da admin admin] The token cache is being invalidate because project 60a8e23a1a3147d3a6efd25d1f3653b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:27.007158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0041c73c-7c5e-4576-82a9-2f6c9b1239da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:27.008273 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0041c73c-7c5e-4576-82a9-2f6c9b1239da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:27.011785 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 997/2005] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:26 2022] DELETE /identity/v3/projects/60a8e23a1a3147d3a6efd25d1f3653b5 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:27.014936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013a898-b7ad-4e95-acfe-0167f11c769e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:27.019684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013a898-b7ad-4e95-acfe-0167f11c769e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:27.019684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013a898-b7ad-4e95-acfe-0167f11c769e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:27.036792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7b212-adfa-4869-bd11-22efdcd921f9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:27.042333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e32d07-2bb0-4946-9184-0ca4929db338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:27.043191 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e32d07-2bb0-4946-9184-0ca4929db338 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:27.043433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e32d07-2bb0-4946-9184-0ca4929db338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:27.043705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e32d07-2bb0-4946-9184-0ca4929db338 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:27.044753 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e32d07-2bb0-4946-9184-0ca4929db338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:27.048513 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e32d07-2bb0-4946-9184-0ca4929db338 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:27.048723 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8013a898-b7ad-4e95-acfe-0167f11c769e None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:27.049561 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8013a898-b7ad-4e95-acfe-0167f11c769e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:27.058573 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1009/2006] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:27.062216 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 998/2007] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:27.070016 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e32d07-2bb0-4946-9184-0ca4929db338 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:27.070650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61540fe-cf95-4bf4-96cb-b38c094e2af3 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:27.071278 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61540fe-cf95-4bf4-96cb-b38c094e2af3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:27.071847 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61540fe-cf95-4bf4-96cb-b38c094e2af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:27.072018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f9e32d07-2bb0-4946-9184-0ca4929db338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:27.092372 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e32d07-2bb0-4946-9184-0ca4929db338 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:27.098043 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb6fce-af1a-4e6c-8ba7-9de0d009a920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:27.108275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb6fce-af1a-4e6c-8ba7-9de0d009a920 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:27.108275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb6fce-af1a-4e6c-8ba7-9de0d009a920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:27.108275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb6fce-af1a-4e6c-8ba7-9de0d009a920 admin admin] PATH_INFO: `/v3/projects/ee462318179e41d2b72893d6936916f8/users/c90f4ac8f19a48509c6497720e97784b/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:27.126288 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb6fce-af1a-4e6c-8ba7-9de0d009a920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee462318179e41d2b72893d6936916f8, user_id=c90f4ac8f19a48509c6497720e97784b, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:27.129097 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb6fce-af1a-4e6c-8ba7-9de0d009a920 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:27.136141 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d61540fe-cf95-4bf4-96cb-b38c094e2af3 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:27.138334 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d61540fe-cf95-4bf4-96cb-b38c094e2af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:27.154258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 999/2008] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:27.175548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61540fe-cf95-4bf4-96cb-b38c094e2af3 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:27.183482 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61540fe-cf95-4bf4-96cb-b38c094e2af3 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:27.190945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046de1b2-b3bc-46b9-b9ba-20f9d716628f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:27.192217 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046de1b2-b3bc-46b9-b9ba-20f9d716628f admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:27.192532 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046de1b2-b3bc-46b9-b9ba-20f9d716628f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:27.192894 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046de1b2-b3bc-46b9-b9ba-20f9d716628f admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:27.193815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046de1b2-b3bc-46b9-b9ba-20f9d716628f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:07:27.194349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046de1b2-b3bc-46b9-b9ba-20f9d716628f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:27.197596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046de1b2-b3bc-46b9-b9ba-20f9d716628f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:27.216461 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1000/2009] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:07:27.221820 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046de1b2-b3bc-46b9-b9ba-20f9d716628f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:27.232770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1010/2010] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:27 2022] PUT /identity/v3/projects/ee462318179e41d2b72893d6936916f8/users/c90f4ac8f19a48509c6497720e97784b/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:27.238701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046de1b2-b3bc-46b9-b9ba-20f9d716628f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:27.238865 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9db86-322e-45c0-90f6-afedd3d463c9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:27.239360 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9db86-322e-45c0-90f6-afedd3d463c9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:27.240077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9db86-322e-45c0-90f6-afedd3d463c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:27.278710 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-acb9db86-322e-45c0-90f6-afedd3d463c9 None None] MFA Rules not processed for user `c90f4ac8f19a48509c6497720e97784b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:27.278710 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-acb9db86-322e-45c0-90f6-afedd3d463c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:27.340682 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead57b7f-d786-4452-916e-536cc614025f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:27.341365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead57b7f-d786-4452-916e-536cc614025f admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:27.341657 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead57b7f-d786-4452-916e-536cc614025f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:27.341976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead57b7f-d786-4452-916e-536cc614025f admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:27.344355 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead57b7f-d786-4452-916e-536cc614025f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:27.345401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead57b7f-d786-4452-916e-536cc614025f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:27.388480 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1011/2011] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:27.403992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9db86-322e-45c0-90f6-afedd3d463c9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:27.407162 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-acb9db86-322e-45c0-90f6-afedd3d463c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:27.458057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1001/2012] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:27 2022] POST /identity/v3/projects => generated 389 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:27.458057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead57b7f-d786-4452-916e-536cc614025f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:27.459311 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ead57b7f-d786-4452-916e-536cc614025f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:27.537904 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9db86-322e-45c0-90f6-afedd3d463c9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:27.554852 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead57b7f-d786-4452-916e-536cc614025f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:27.560842 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e62b1-0c6b-4dcb-b50e-df82caaf0ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:27.561787 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e62b1-0c6b-4dcb-b50e-df82caaf0ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:27.564535 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e62b1-0c6b-4dcb-b50e-df82caaf0ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:27.565255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e62b1-0c6b-4dcb-b50e-df82caaf0ce8 admin admin] PATH_INFO: `/v3/projects/29731a9b73fe4c968139d735e3f414e4` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:27.566725 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e62b1-0c6b-4dcb-b50e-df82caaf0ce8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29731a9b73fe4c968139d735e3f414e4)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:27.569669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e62b1-0c6b-4dcb-b50e-df82caaf0ce8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:27.608684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e409742-9e3f-4dbb-8be5-3568295cc8a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b05d0e65e9645f3810a6d96c8fed9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:27.609346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e409742-9e3f-4dbb-8be5-3568295cc8a9 service glance] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:27.609636 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e409742-9e3f-4dbb-8be5-3568295cc8a9 service glance] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:27.609958 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e409742-9e3f-4dbb-8be5-3568295cc8a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:27.684271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e409742-9e3f-4dbb-8be5-3568295cc8a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:27.685607 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e409742-9e3f-4dbb-8be5-3568295cc8a9 service glance] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:27.696358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1012/2013] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:07:27.697283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e409742-9e3f-4dbb-8be5-3568295cc8a9 service glance] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:27.710197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e409742-9e3f-4dbb-8be5-3568295cc8a9 service glance] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:27.719599 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-269e62b1-0c6b-4dcb-b50e-df82caaf0ce8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29731a9b73fe4c968139d735e3f414e4', 'request_id': 'req-269e62b1-0c6b-4dcb-b50e-df82caaf0ce8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:27.790353 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcaffed-eef3-4ea8-9cb8-938960b514f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:27.790982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcaffed-eef3-4ea8-9cb8-938960b514f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:27.791266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcaffed-eef3-4ea8-9cb8-938960b514f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:27.791545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcaffed-eef3-4ea8-9cb8-938960b514f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:27.792179 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcaffed-eef3-4ea8-9cb8-938960b514f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:27.794772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-269e62b1-0c6b-4dcb-b50e-df82caaf0ce8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29731a9b73fe4c968139d735e3f414e4', 'request_id': 'req-269e62b1-0c6b-4dcb-b50e-df82caaf0ce8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:27.795191 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcaffed-eef3-4ea8-9cb8-938960b514f3 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:27.803662 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-269e62b1-0c6b-4dcb-b50e-df82caaf0ce8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29731a9b73fe4c968139d735e3f414e4', 'request_id': 'req-269e62b1-0c6b-4dcb-b50e-df82caaf0ce8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:27.820474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-269e62b1-0c6b-4dcb-b50e-df82caaf0ce8 admin admin] The token cache is being invalidate because project 29731a9b73fe4c968139d735e3f414e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:27.820681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-269e62b1-0c6b-4dcb-b50e-df82caaf0ce8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:27.821377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-269e62b1-0c6b-4dcb-b50e-df82caaf0ce8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:27.823264 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1002/2014] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:27 2022] DELETE /identity/v3/projects/29731a9b73fe4c968139d735e3f414e4 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:27.832022 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e310a-c638-4d81-8b86-a1fb6ca9b029 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:27.832022 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e310a-c638-4d81-8b86-a1fb6ca9b029 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:27.832022 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e310a-c638-4d81-8b86-a1fb6ca9b029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:27.852704 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b81e310a-c638-4d81-8b86-a1fb6ca9b029 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:27.854084 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b81e310a-c638-4d81-8b86-a1fb6ca9b029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:27.870280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1003/2015] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:27.887201 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b039d8-4311-4642-a372-ab6bcb6d21f8 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:27.888189 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b039d8-4311-4642-a372-ab6bcb6d21f8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:27.888189 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b039d8-4311-4642-a372-ab6bcb6d21f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:27.909460 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1013/2016] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:27 2022] POST /identity/v3/users => generated 295 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:27.910649 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcaffed-eef3-4ea8-9cb8-938960b514f3 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:27.912922 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4bcaffed-eef3-4ea8-9cb8-938960b514f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:27.919724 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a6b039d8-4311-4642-a372-ab6bcb6d21f8 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:27.921104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a6b039d8-4311-4642-a372-ab6bcb6d21f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:27.926200 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcaffed-eef3-4ea8-9cb8-938960b514f3 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:27.931701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1004/2017] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:27.931975 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51d91d-a28d-4c6c-a732-d0b4f4c02325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:27.933110 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51d91d-a28d-4c6c-a732-d0b4f4c02325 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:27.933470 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51d91d-a28d-4c6c-a732-d0b4f4c02325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:27.933716 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51d91d-a28d-4c6c-a732-d0b4f4c02325 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:27.934397 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51d91d-a28d-4c6c-a732-d0b4f4c02325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:27.939060 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b039d8-4311-4642-a372-ab6bcb6d21f8 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:27.939304 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51d91d-a28d-4c6c-a732-d0b4f4c02325 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:27.948173 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b039d8-4311-4642-a372-ab6bcb6d21f8 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:27.952235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1014/2018] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:07:27.953140 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdb8ae-8207-4097-86c1-033c5db14a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:27.953380 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51d91d-a28d-4c6c-a732-d0b4f4c02325 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:27.953795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdb8ae-8207-4097-86c1-033c5db14a35 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:27.954117 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdb8ae-8207-4097-86c1-033c5db14a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:27.954398 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdb8ae-8207-4097-86c1-033c5db14a35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:27.955041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdb8ae-8207-4097-86c1-033c5db14a35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:07:27.955342 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdb8ae-8207-4097-86c1-033c5db14a35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:27.956944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdb8ae-8207-4097-86c1-033c5db14a35 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:27.964522 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51d91d-a28d-4c6c-a732-d0b4f4c02325 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:27.969852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c11412-6e9b-44e3-bd1c-1898bbe05fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:27.970584 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c11412-6e9b-44e3-bd1c-1898bbe05fba admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:27.970946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c11412-6e9b-44e3-bd1c-1898bbe05fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:27.971377 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c11412-6e9b-44e3-bd1c-1898bbe05fba admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:27.971587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1005/2019] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 12:07:27.972197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c11412-6e9b-44e3-bd1c-1898bbe05fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:27.973953 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c11412-6e9b-44e3-bd1c-1898bbe05fba admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:27.976258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdb8ae-8207-4097-86c1-033c5db14a35 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:27.989286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdb8ae-8207-4097-86c1-033c5db14a35 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:27.989520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1015/2020] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:27.993887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c11412-6e9b-44e3-bd1c-1898bbe05fba admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:27.999746 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f408181-77f7-4966-a5fb-e661fc98d273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:28.001066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f408181-77f7-4966-a5fb-e661fc98d273 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:28.001441 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f408181-77f7-4966-a5fb-e661fc98d273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:28.002262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f408181-77f7-4966-a5fb-e661fc98d273 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:28.003076 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f408181-77f7-4966-a5fb-e661fc98d273 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:28.007061 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f408181-77f7-4966-a5fb-e661fc98d273 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:28.007519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c11412-6e9b-44e3-bd1c-1898bbe05fba admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:28.017590 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e62cc6-fb39-481f-a9d6-d49f8e584f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:28.018326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e62cc6-fb39-481f-a9d6-d49f8e584f2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:28.018602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e62cc6-fb39-481f-a9d6-d49f8e584f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:28.018884 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e62cc6-fb39-481f-a9d6-d49f8e584f2f admin admin] PATH_INFO: `/v3/projects/03d2c576e35b47059eab93b26773fd09/users/127f2410cd5f4c2686703ac80c969c6f/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:28.050994 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e62cc6-fb39-481f-a9d6-d49f8e584f2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=03d2c576e35b47059eab93b26773fd09, user_id=127f2410cd5f4c2686703ac80c969c6f, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:28.054693 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e62cc6-fb39-481f-a9d6-d49f8e584f2f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:28.097970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1006/2021] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:27 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:07:28.101614 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f408181-77f7-4966-a5fb-e661fc98d273 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:28.133956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1016/2022] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:27 2022] PUT /identity/v3/projects/03d2c576e35b47059eab93b26773fd09/users/127f2410cd5f4c2686703ac80c969c6f/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:07:28.145222 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fe909-7db5-43a8-a42b-e83306be76ce None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:28.146126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fe909-7db5-43a8-a42b-e83306be76ce None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:28.146744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fe909-7db5-43a8-a42b-e83306be76ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:28.194965 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6f0fe909-7db5-43a8-a42b-e83306be76ce None None] MFA Rules not processed for user `127f2410cd5f4c2686703ac80c969c6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:28.196253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6f0fe909-7db5-43a8-a42b-e83306be76ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:28.222853 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f408181-77f7-4966-a5fb-e661fc98d273 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:28.282171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1017/2023] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:28.291685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fe909-7db5-43a8-a42b-e83306be76ce None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:28.292453 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912a7ed-3391-4c6d-8486-5351886fa556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:28.293074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912a7ed-3391-4c6d-8486-5351886fa556 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:28.293355 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912a7ed-3391-4c6d-8486-5351886fa556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:28.293633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912a7ed-3391-4c6d-8486-5351886fa556 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:28.294311 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912a7ed-3391-4c6d-8486-5351886fa556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:28.296307 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912a7ed-3391-4c6d-8486-5351886fa556 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:28.300126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fe909-7db5-43a8-a42b-e83306be76ce None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:28.305569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42bc3e-6516-45a4-872d-b998cb34b772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:28.306228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42bc3e-6516-45a4-872d-b998cb34b772 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:28.306511 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42bc3e-6516-45a4-872d-b998cb34b772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:28.306790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42bc3e-6516-45a4-872d-b998cb34b772 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:28.307441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc42bc3e-6516-45a4-872d-b998cb34b772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:28.309351 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc42bc3e-6516-45a4-872d-b998cb34b772 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:28.366571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:28.435920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1018/2024] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:28 2022] POST /identity/v3/projects => generated 387 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:07:28.436992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42bc3e-6516-45a4-872d-b998cb34b772 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:28.437281 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1007/2025] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:28 2022] POST /identity/v3/users => generated 293 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:07:28.438642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912a7ed-3391-4c6d-8486-5351886fa556 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:28.451295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912a7ed-3391-4c6d-8486-5351886fa556 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:28.469573 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:28.535816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42bc3e-6516-45a4-872d-b998cb34b772 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:28.541996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb201129-dc7c-4157-82e1-566b2a7f1d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:28.542654 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb201129-dc7c-4157-82e1-566b2a7f1d78 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:28.543009 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb201129-dc7c-4157-82e1-566b2a7f1d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:28.543674 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb201129-dc7c-4157-82e1-566b2a7f1d78 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:28.544052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb201129-dc7c-4157-82e1-566b2a7f1d78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:28.547070 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb201129-dc7c-4157-82e1-566b2a7f1d78 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:28.554589 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d83ea1-d4a8-4edc-9369-8904e0895c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:28.555230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d83ea1-d4a8-4edc-9369-8904e0895c0e admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:28.555523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d83ea1-d4a8-4edc-9369-8904e0895c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:28.555808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d83ea1-d4a8-4edc-9369-8904e0895c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:28.556480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d83ea1-d4a8-4edc-9369-8904e0895c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:28.558448 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d83ea1-d4a8-4edc-9369-8904e0895c0e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:28.573022 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1008/2026] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:28.575073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d83ea1-d4a8-4edc-9369-8904e0895c0e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:28.585769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d83ea1-d4a8-4edc-9369-8904e0895c0e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:28.590668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f49d18-0c5f-49e1-b087-bf0b07d2edee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:28.591283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f49d18-0c5f-49e1-b087-bf0b07d2edee admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:28.591570 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f49d18-0c5f-49e1-b087-bf0b07d2edee admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:28.591848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f49d18-0c5f-49e1-b087-bf0b07d2edee admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:28.592493 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f49d18-0c5f-49e1-b087-bf0b07d2edee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:28.594874 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f49d18-0c5f-49e1-b087-bf0b07d2edee admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:28.606994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1009/2027] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:07:28.611327 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f49d18-0c5f-49e1-b087-bf0b07d2edee admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:28.621914 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f49d18-0c5f-49e1-b087-bf0b07d2edee admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:28.626818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ef036-683a-465e-a123-c89d85a7edb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:28.627494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ef036-683a-465e-a123-c89d85a7edb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:28.628983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ef036-683a-465e-a123-c89d85a7edb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:28.628983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ef036-683a-465e-a123-c89d85a7edb5 admin admin] PATH_INFO: `/v3/projects/bef4d06eb14b4a2482dd3fea2339f40d/users/220c19e1cbfd4d0eb9602f8390117ee7/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:28.645033 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ef036-683a-465e-a123-c89d85a7edb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef4d06eb14b4a2482dd3fea2339f40d, user_id=220c19e1cbfd4d0eb9602f8390117ee7, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:28.647861 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ef036-683a-465e-a123-c89d85a7edb5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:28.716665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1019/2028] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:28 2022] POST /identity/v3/users => generated 300 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:28.723153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb201129-dc7c-4157-82e1-566b2a7f1d78 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:28.733405 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb201129-dc7c-4157-82e1-566b2a7f1d78 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:28.743373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1010/2029] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:28 2022] PUT /identity/v3/projects/bef4d06eb14b4a2482dd3fea2339f40d/users/220c19e1cbfd4d0eb9602f8390117ee7/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:07:28.755296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6639d1a-d036-4e39-994d-98d0313ee755 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:28.755803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6639d1a-d036-4e39-994d-98d0313ee755 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:28.756249 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6639d1a-d036-4e39-994d-98d0313ee755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:28.798742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e6639d1a-d036-4e39-994d-98d0313ee755 None None] MFA Rules not processed for user `220c19e1cbfd4d0eb9602f8390117ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:28.800984 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e6639d1a-d036-4e39-994d-98d0313ee755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:28.820968 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cf532-99f3-4d0b-846a-728e0adc3aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:28.821842 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cf532-99f3-4d0b-846a-728e0adc3aae admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:28.822250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cf532-99f3-4d0b-846a-728e0adc3aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:28.822646 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cf532-99f3-4d0b-846a-728e0adc3aae admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:28.823404 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cf532-99f3-4d0b-846a-728e0adc3aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:28.825275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cf532-99f3-4d0b-846a-728e0adc3aae admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:28.841096 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1020/2030] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:07:28.845362 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cf532-99f3-4d0b-846a-728e0adc3aae admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:28.856412 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cf532-99f3-4d0b-846a-728e0adc3aae admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:28.862201 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad24459-fe0f-4995-adc1-48e5eb604870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:28.862984 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad24459-fe0f-4995-adc1-48e5eb604870 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:28.863257 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad24459-fe0f-4995-adc1-48e5eb604870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:28.863532 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad24459-fe0f-4995-adc1-48e5eb604870 admin admin] PATH_INFO: `/v3/projects/f64eb2709b81460eb119cfe00fdb9274/users/0a489156cefe45b58201db40a558b9fb/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:28.881829 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad24459-fe0f-4995-adc1-48e5eb604870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64eb2709b81460eb119cfe00fdb9274, user_id=0a489156cefe45b58201db40a558b9fb, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:28.885406 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad24459-fe0f-4995-adc1-48e5eb604870 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:28.895182 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1011/2031] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:28.901960 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6639d1a-d036-4e39-994d-98d0313ee755 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:28.917521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6639d1a-d036-4e39-994d-98d0313ee755 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:28.923063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cfaa87-9556-463f-a176-20876c361f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:28.923695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfaa87-9556-463f-a176-20876c361f8b admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:28.923985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfaa87-9556-463f-a176-20876c361f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:28.924265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfaa87-9556-463f-a176-20876c361f8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:28.924948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cfaa87-9556-463f-a176-20876c361f8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:28.927656 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cfaa87-9556-463f-a176-20876c361f8b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:28.979899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1021/2032] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:28 2022] PUT /identity/v3/projects/f64eb2709b81460eb119cfe00fdb9274/users/0a489156cefe45b58201db40a558b9fb/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:07:28.983642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad24459-fe0f-4995-adc1-48e5eb604870 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:29.014198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1012/2033] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:28 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:07:29.019155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cfaa87-9556-463f-a176-20876c361f8b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:29.059939 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad24459-fe0f-4995-adc1-48e5eb604870 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:29.099915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cfaa87-9556-463f-a176-20876c361f8b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:29.105320 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91534747-8dc0-4143-902b-c32e7415365f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:29.105983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91534747-8dc0-4143-902b-c32e7415365f admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:29.106271 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91534747-8dc0-4143-902b-c32e7415365f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:29.106573 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91534747-8dc0-4143-902b-c32e7415365f admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:29.107230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91534747-8dc0-4143-902b-c32e7415365f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:29.109228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91534747-8dc0-4143-902b-c32e7415365f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:29.134985 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e6353-c47b-4d2d-afdd-f91cbb52cfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:29.135604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e6353-c47b-4d2d-afdd-f91cbb52cfe0 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:29.135951 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e6353-c47b-4d2d-afdd-f91cbb52cfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:29.136227 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e6353-c47b-4d2d-afdd-f91cbb52cfe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:29.136879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e6353-c47b-4d2d-afdd-f91cbb52cfe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:29.138526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e6353-c47b-4d2d-afdd-f91cbb52cfe0 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:29.150281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1022/2034] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:07:29.154395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e6353-c47b-4d2d-afdd-f91cbb52cfe0 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:29.164457 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e6353-c47b-4d2d-afdd-f91cbb52cfe0 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:29.169293 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c09707-5ba6-467b-9bf5-f380e69025e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:29.169930 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c09707-5ba6-467b-9bf5-f380e69025e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:29.170202 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c09707-5ba6-467b-9bf5-f380e69025e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:29.170485 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c09707-5ba6-467b-9bf5-f380e69025e3 admin admin] PATH_INFO: `/v3/projects/f64eb2709b81460eb119cfe00fdb9274/users/0a489156cefe45b58201db40a558b9fb/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:29.172455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c09707-5ba6-467b-9bf5-f380e69025e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64eb2709b81460eb119cfe00fdb9274, user_id=0a489156cefe45b58201db40a558b9fb, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:29.175190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c09707-5ba6-467b-9bf5-f380e69025e3 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:29.185768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1023/2035] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:29 2022] PUT /identity/v3/projects/f64eb2709b81460eb119cfe00fdb9274/users/0a489156cefe45b58201db40a558b9fb/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:07:29.196717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f09d8-0b10-49ac-8098-c235bc207878 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:29.197002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f09d8-0b10-49ac-8098-c235bc207878 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:29.197414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f09d8-0b10-49ac-8098-c235bc207878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:29.206664 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:29.226640 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-926f09d8-0b10-49ac-8098-c235bc207878 None None] MFA Rules not processed for user `0a489156cefe45b58201db40a558b9fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:29.228036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-926f09d8-0b10-49ac-8098-c235bc207878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:29.240966 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1013/2036] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:29 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:07:29.245297 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91534747-8dc0-4143-902b-c32e7415365f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:29.256762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91534747-8dc0-4143-902b-c32e7415365f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:29.266101 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:29.313051 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1024/2037] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:29.325762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f09d8-0b10-49ac-8098-c235bc207878 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:29.327431 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-926f09d8-0b10-49ac-8098-c235bc207878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:29.334940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361397aa-6c7b-44e3-a8cb-9fc05e9f9e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:29.335976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361397aa-6c7b-44e3-a8cb-9fc05e9f9e81 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:29.336390 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361397aa-6c7b-44e3-a8cb-9fc05e9f9e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:29.336668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361397aa-6c7b-44e3-a8cb-9fc05e9f9e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:29.337590 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361397aa-6c7b-44e3-a8cb-9fc05e9f9e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:29.339524 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361397aa-6c7b-44e3-a8cb-9fc05e9f9e81 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:29.351892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1014/2038] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:29.355976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361397aa-6c7b-44e3-a8cb-9fc05e9f9e81 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:29.368078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361397aa-6c7b-44e3-a8cb-9fc05e9f9e81 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:29.375121 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559c2e2-0893-49b6-a3cf-72139c44956f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:29.376087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559c2e2-0893-49b6-a3cf-72139c44956f admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:29.376456 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559c2e2-0893-49b6-a3cf-72139c44956f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:29.376845 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559c2e2-0893-49b6-a3cf-72139c44956f admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:29.377619 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1559c2e2-0893-49b6-a3cf-72139c44956f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:29.379408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1559c2e2-0893-49b6-a3cf-72139c44956f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:29.391387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f09d8-0b10-49ac-8098-c235bc207878 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:29.393825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1015/2039] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:29.397676 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559c2e2-0893-49b6-a3cf-72139c44956f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:29.398593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53485b-a5b2-431e-9821-bc57a5f7ba7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:29.399718 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53485b-a5b2-431e-9821-bc57a5f7ba7b service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:29.406017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53485b-a5b2-431e-9821-bc57a5f7ba7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:29.406017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53485b-a5b2-431e-9821-bc57a5f7ba7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:29.406230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:29.411477 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b53485b-a5b2-431e-9821-bc57a5f7ba7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:29.412890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559c2e2-0893-49b6-a3cf-72139c44956f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:29.413013 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b53485b-a5b2-431e-9821-bc57a5f7ba7b service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:29.419901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e0073-fc9c-4ac0-a1ec-ac527b22df66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:29.420825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e0073-fc9c-4ac0-a1ec-ac527b22df66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:29.421137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e0073-fc9c-4ac0-a1ec-ac527b22df66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:29.422098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e0073-fc9c-4ac0-a1ec-ac527b22df66 admin admin] PATH_INFO: `/v3/projects/a983b75ccf294ffea19d55b14ee2a015/users/ae8f3f1d99b644c78e3a3cb6637ba43b/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:29.424960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1025/2040] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:07:29.442658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e0073-fc9c-4ac0-a1ec-ac527b22df66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a983b75ccf294ffea19d55b14ee2a015, user_id=ae8f3f1d99b644c78e3a3cb6637ba43b, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:29.445427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e0073-fc9c-4ac0-a1ec-ac527b22df66 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:29.458428 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:29.532709 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1016/2041] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:29 2022] PUT /identity/v3/projects/a983b75ccf294ffea19d55b14ee2a015/users/ae8f3f1d99b644c78e3a3cb6637ba43b/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:07:29.535487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab530e1-da4c-4dbc-b8ea-0f46f4eb45b1 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:29.535990 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab530e1-da4c-4dbc-b8ea-0f46f4eb45b1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:29.536415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab530e1-da4c-4dbc-b8ea-0f46f4eb45b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:29.572765 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-dab530e1-da4c-4dbc-b8ea-0f46f4eb45b1 None None] MFA Rules not processed for user `ae8f3f1d99b644c78e3a3cb6637ba43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:29.574373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-dab530e1-da4c-4dbc-b8ea-0f46f4eb45b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:29.588186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e0073-fc9c-4ac0-a1ec-ac527b22df66 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:29.608844 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:29.664914 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e0073-fc9c-4ac0-a1ec-ac527b22df66 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:29.675184 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1026/2042] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:29.677012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2be4b5-ba78-4924-aa6e-6ab9961d1ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:29.678921 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2be4b5-ba78-4924-aa6e-6ab9961d1ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:29.678921 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2be4b5-ba78-4924-aa6e-6ab9961d1ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:29.678921 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2be4b5-ba78-4924-aa6e-6ab9961d1ec3 admin admin] PATH_INFO: `/v3/users/127f2410cd5f4c2686703ac80c969c6f` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:29.681785 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2be4b5-ba78-4924-aa6e-6ab9961d1ec3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=127f2410cd5f4c2686703ac80c969c6f)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:29.683828 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab530e1-da4c-4dbc-b8ea-0f46f4eb45b1 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:29.688536 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2be4b5-ba78-4924-aa6e-6ab9961d1ec3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:29.697201 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab530e1-da4c-4dbc-b8ea-0f46f4eb45b1 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:29.707040 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87f549-d1a2-481d-98ce-f7a55955a9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:29.708150 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87f549-d1a2-481d-98ce-f7a55955a9c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:29.708948 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87f549-d1a2-481d-98ce-f7a55955a9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:29.709693 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87f549-d1a2-481d-98ce-f7a55955a9c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:29.714724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:29.719021 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87f549-d1a2-481d-98ce-f7a55955a9c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:29.719021 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87f549-d1a2-481d-98ce-f7a55955a9c0 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:29.826734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1027/2043] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:29 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:29.830547 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87f549-d1a2-481d-98ce-f7a55955a9c0 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:29.895777 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87f549-d1a2-481d-98ce-f7a55955a9c0 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:29.902005 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19970d80-7404-48d5-8747-f9a55da6a5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:29.903281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19970d80-7404-48d5-8747-f9a55da6a5af admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:29.903643 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19970d80-7404-48d5-8747-f9a55da6a5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:29.904118 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19970d80-7404-48d5-8747-f9a55da6a5af admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:29.904912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19970d80-7404-48d5-8747-f9a55da6a5af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:29.907588 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19970d80-7404-48d5-8747-f9a55da6a5af admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:29.925103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e2be4b5-ba78-4924-aa6e-6ab9961d1ec3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127f2410cd5f4c2686703ac80c969c6f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:29.927344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:29.937568 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e2be4b5-ba78-4924-aa6e-6ab9961d1ec3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127f2410cd5f4c2686703ac80c969c6f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:29.944827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e2be4b5-ba78-4924-aa6e-6ab9961d1ec3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127f2410cd5f4c2686703ac80c969c6f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:30.006675 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1028/2044] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:29 2022] POST /identity/v3/users => generated 299 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:07:30.010725 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19970d80-7404-48d5-8747-f9a55da6a5af admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:30.021925 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19970d80-7404-48d5-8747-f9a55da6a5af admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:30.024816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e2be4b5-ba78-4924-aa6e-6ab9961d1ec3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127f2410cd5f4c2686703ac80c969c6f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:30.024816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:30.025505 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e2be4b5-ba78-4924-aa6e-6ab9961d1ec3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127f2410cd5f4c2686703ac80c969c6f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:30.026944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c8a35-fdff-433a-a9c9-de799add075a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:30.027548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c8a35-fdff-433a-a9c9-de799add075a admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:30.027856 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c8a35-fdff-433a-a9c9-de799add075a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:30.028407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c8a35-fdff-433a-a9c9-de799add075a admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:30.028774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c8a35-fdff-433a-a9c9-de799add075a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:30.030431 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c8a35-fdff-433a-a9c9-de799add075a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:30.034488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1017/2045] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:29 2022] DELETE /identity/v3/users/127f2410cd5f4c2686703ac80c969c6f => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:07:30.039245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d306930-88a8-4576-a209-9792a47f66b9 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:30.039578 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d306930-88a8-4576-a209-9792a47f66b9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:30.039895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d306930-88a8-4576-a209-9792a47f66b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:30.043139 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1029/2046] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:07:30.050838 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c8a35-fdff-433a-a9c9-de799add075a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:30.052819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2f4c8a35-fdff-433a-a9c9-de799add075a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:30.060035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-7d306930-88a8-4576-a209-9792a47f66b9 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:30.061261 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7d306930-88a8-4576-a209-9792a47f66b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:30.068308 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c8a35-fdff-433a-a9c9-de799add075a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:30.131278 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1018/2047] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:30.139787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5b61c-eaa2-4384-ad8b-c7c354e899fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:30.141043 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5b61c-eaa2-4384-ad8b-c7c354e899fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:30.141512 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5b61c-eaa2-4384-ad8b-c7c354e899fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:30.141805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5b61c-eaa2-4384-ad8b-c7c354e899fb admin admin] PATH_INFO: `/v3/projects/1d7ecf8648244d9d9c8237c47775fc7e/users/eb4b5843d2e746c2b3c8513a3921a18c/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:30.147631 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d306930-88a8-4576-a209-9792a47f66b9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:30.149590 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7d306930-88a8-4576-a209-9792a47f66b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:30.158279 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:30.164990 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5b61c-eaa2-4384-ad8b-c7c354e899fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d7ecf8648244d9d9c8237c47775fc7e, user_id=eb4b5843d2e746c2b3c8513a3921a18c, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:30.167683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5b61c-eaa2-4384-ad8b-c7c354e899fb admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:30.183408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:30.211292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d306930-88a8-4576-a209-9792a47f66b9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:30.214913 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112750c-83a4-47f6-8bcc-6816f64dea04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:30.215525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112750c-83a4-47f6-8bcc-6816f64dea04 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:30.215805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112750c-83a4-47f6-8bcc-6816f64dea04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:30.216093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112750c-83a4-47f6-8bcc-6816f64dea04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:30.225675 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112750c-83a4-47f6-8bcc-6816f64dea04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:30.227144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112750c-83a4-47f6-8bcc-6816f64dea04 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:30.236512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1019/2048] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:30.240763 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1030/2049] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:30 2022] PUT /identity/v3/projects/1d7ecf8648244d9d9c8237c47775fc7e/users/eb4b5843d2e746c2b3c8513a3921a18c/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:07:30.244707 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112750c-83a4-47f6-8bcc-6816f64dea04 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:30.254374 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112750c-83a4-47f6-8bcc-6816f64dea04 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:30.315727 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14873c70-c4c5-48a1-b1bf-1f9393f8989a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:30.316388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14873c70-c4c5-48a1-b1bf-1f9393f8989a admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:30.316672 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14873c70-c4c5-48a1-b1bf-1f9393f8989a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:30.316950 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14873c70-c4c5-48a1-b1bf-1f9393f8989a admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:30.317592 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14873c70-c4c5-48a1-b1bf-1f9393f8989a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:30.319361 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14873c70-c4c5-48a1-b1bf-1f9393f8989a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:30.330365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1020/2050] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:07:30.335242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5b61c-eaa2-4384-ad8b-c7c354e899fb admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:30.346303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5b61c-eaa2-4384-ad8b-c7c354e899fb admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:30.351029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d2944-0ac2-4fad-9605-5fa65fc53cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:30.351677 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d2944-0ac2-4fad-9605-5fa65fc53cee admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:30.351957 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d2944-0ac2-4fad-9605-5fa65fc53cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:30.352230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d2944-0ac2-4fad-9605-5fa65fc53cee admin admin] PATH_INFO: `/v3/projects/1d7ecf8648244d9d9c8237c47775fc7e/users/eb4b5843d2e746c2b3c8513a3921a18c/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:30.354164 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d2944-0ac2-4fad-9605-5fa65fc53cee admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d7ecf8648244d9d9c8237c47775fc7e, user_id=eb4b5843d2e746c2b3c8513a3921a18c, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:30.358945 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d2944-0ac2-4fad-9605-5fa65fc53cee admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:30.366892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1031/2051] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:30 2022] PUT /identity/v3/projects/1d7ecf8648244d9d9c8237c47775fc7e/users/eb4b5843d2e746c2b3c8513a3921a18c/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:30.377070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8403c52-4af0-47a5-9dcf-2ffd62827232 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:30.377364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8403c52-4af0-47a5-9dcf-2ffd62827232 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:30.377646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8403c52-4af0-47a5-9dcf-2ffd62827232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:30.377752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:30.405467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d8403c52-4af0-47a5-9dcf-2ffd62827232 None None] MFA Rules not processed for user `eb4b5843d2e746c2b3c8513a3921a18c`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:30.407256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d8403c52-4af0-47a5-9dcf-2ffd62827232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:30.419223 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:30.483894 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1021/2052] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:30.494817 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d2944-0ac2-4fad-9605-5fa65fc53cee admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:30.496527 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f46d2944-0ac2-4fad-9605-5fa65fc53cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:30.557434 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d2944-0ac2-4fad-9605-5fa65fc53cee admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:30.561429 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561c90e-f2b8-41d6-a728-dfb78dee1ebb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:30.562564 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561c90e-f2b8-41d6-a728-dfb78dee1ebb service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:30.562850 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561c90e-f2b8-41d6-a728-dfb78dee1ebb service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:30.563123 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561c90e-f2b8-41d6-a728-dfb78dee1ebb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:30.573509 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561c90e-f2b8-41d6-a728-dfb78dee1ebb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:30.574976 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561c90e-f2b8-41d6-a728-dfb78dee1ebb service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:30.585156 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1032/2053] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:30.612862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:30.642083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8403c52-4af0-47a5-9dcf-2ffd62827232 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:30.644091 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d8403c52-4af0-47a5-9dcf-2ffd62827232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:30.738866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8403c52-4af0-47a5-9dcf-2ffd62827232 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:30.743569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e8be0-89a2-4359-8ee0-09791110ae34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:30.744207 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e8be0-89a2-4359-8ee0-09791110ae34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:30.744510 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e8be0-89a2-4359-8ee0-09791110ae34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:30.744794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e8be0-89a2-4359-8ee0-09791110ae34 admin admin] PATH_INFO: `/v3/projects/03d2c576e35b47059eab93b26773fd09` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:30.745961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e8be0-89a2-4359-8ee0-09791110ae34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03d2c576e35b47059eab93b26773fd09)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:30.747933 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e8be0-89a2-4359-8ee0-09791110ae34 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:30.883307 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:30.892617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-f23e8be0-89a2-4359-8ee0-09791110ae34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03d2c576e35b47059eab93b26773fd09', 'request_id': 'req-f23e8be0-89a2-4359-8ee0-09791110ae34'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:30.972670 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-f23e8be0-89a2-4359-8ee0-09791110ae34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03d2c576e35b47059eab93b26773fd09', 'request_id': 'req-f23e8be0-89a2-4359-8ee0-09791110ae34'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:30.981774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-f23e8be0-89a2-4359-8ee0-09791110ae34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03d2c576e35b47059eab93b26773fd09', 'request_id': 'req-f23e8be0-89a2-4359-8ee0-09791110ae34'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:30.995023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-f23e8be0-89a2-4359-8ee0-09791110ae34 admin admin] The token cache is being invalidate because project 03d2c576e35b47059eab93b26773fd09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:30.995505 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-f23e8be0-89a2-4359-8ee0-09791110ae34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:30.996180 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-f23e8be0-89a2-4359-8ee0-09791110ae34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:30.997846 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1022/2054] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:30 2022] DELETE /identity/v3/projects/03d2c576e35b47059eab93b26773fd09 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:31.001887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561c90e-f2b8-41d6-a728-dfb78dee1ebb service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:31.013419 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561c90e-f2b8-41d6-a728-dfb78dee1ebb service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:31.078360 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853203c-526a-4522-842e-21d605c228a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:31.079013 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853203c-526a-4522-842e-21d605c228a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:31.079297 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853203c-526a-4522-842e-21d605c228a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:31.079578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853203c-526a-4522-842e-21d605c228a1 admin admin] PATH_INFO: `/v3/users/0a489156cefe45b58201db40a558b9fb` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:31.080746 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853203c-526a-4522-842e-21d605c228a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a489156cefe45b58201db40a558b9fb)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:31.082821 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853203c-526a-4522-842e-21d605c228a1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:31.271788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3853203c-526a-4522-842e-21d605c228a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a489156cefe45b58201db40a558b9fb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:31.283987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3853203c-526a-4522-842e-21d605c228a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a489156cefe45b58201db40a558b9fb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:31.292275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3853203c-526a-4522-842e-21d605c228a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a489156cefe45b58201db40a558b9fb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:31.377687 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3853203c-526a-4522-842e-21d605c228a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a489156cefe45b58201db40a558b9fb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:31.378445 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3853203c-526a-4522-842e-21d605c228a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a489156cefe45b58201db40a558b9fb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:31.388777 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1033/2055] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:31 2022] DELETE /identity/v3/users/0a489156cefe45b58201db40a558b9fb => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:31.794622 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e8be0-89a2-4359-8ee0-09791110ae34 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:31.797369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f23e8be0-89a2-4359-8ee0-09791110ae34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:31.802657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853203c-526a-4522-842e-21d605c228a1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:31.805192 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3853203c-526a-4522-842e-21d605c228a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:31.839791 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:31.973731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e8be0-89a2-4359-8ee0-09791110ae34 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:31.977429 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853203c-526a-4522-842e-21d605c228a1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:31.981400 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25989e7-eb85-494a-9fc7-376545dcfbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:31.982219 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25989e7-eb85-494a-9fc7-376545dcfbd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:31.982625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25989e7-eb85-494a-9fc7-376545dcfbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:31.982993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25989e7-eb85-494a-9fc7-376545dcfbd6 admin admin] PATH_INFO: `/v3/projects/f64eb2709b81460eb119cfe00fdb9274` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:31.985027 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25989e7-eb85-494a-9fc7-376545dcfbd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f64eb2709b81460eb119cfe00fdb9274)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:31.987845 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25989e7-eb85-494a-9fc7-376545dcfbd6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:32.066998 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb72eac-be87-4f4d-bf22-2e2d87f81532 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:32.067703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb72eac-be87-4f4d-bf22-2e2d87f81532 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:32.067995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb72eac-be87-4f4d-bf22-2e2d87f81532 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:32.068276 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb72eac-be87-4f4d-bf22-2e2d87f81532 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:32.070407 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7bb72eac-be87-4f4d-bf22-2e2d87f81532 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:32.146955 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb72eac-be87-4f4d-bf22-2e2d87f81532 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:32.148477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb72eac-be87-4f4d-bf22-2e2d87f81532 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:32.169362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1023/2056] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:32.244950 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d25989e7-eb85-494a-9fc7-376545dcfbd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f64eb2709b81460eb119cfe00fdb9274', 'request_id': 'req-d25989e7-eb85-494a-9fc7-376545dcfbd6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:32.337717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d25989e7-eb85-494a-9fc7-376545dcfbd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f64eb2709b81460eb119cfe00fdb9274', 'request_id': 'req-d25989e7-eb85-494a-9fc7-376545dcfbd6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:32.344902 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d25989e7-eb85-494a-9fc7-376545dcfbd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f64eb2709b81460eb119cfe00fdb9274', 'request_id': 'req-d25989e7-eb85-494a-9fc7-376545dcfbd6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:32.357132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d25989e7-eb85-494a-9fc7-376545dcfbd6 admin admin] The token cache is being invalidate because project f64eb2709b81460eb119cfe00fdb9274 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:32.357440 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d25989e7-eb85-494a-9fc7-376545dcfbd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:32.358139 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d25989e7-eb85-494a-9fc7-376545dcfbd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:32.360174 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1034/2057] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:31 2022] DELETE /identity/v3/projects/f64eb2709b81460eb119cfe00fdb9274 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:32.365677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c56a5-8d0a-4507-9c95-43707715728f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:32.365980 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c56a5-8d0a-4507-9c95-43707715728f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:32.366267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c56a5-8d0a-4507-9c95-43707715728f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:32.387644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4c4c56a5-8d0a-4507-9c95-43707715728f None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:32.389937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4c4c56a5-8d0a-4507-9c95-43707715728f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:32.453386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:32.483293 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1024/2058] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:32.492430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ad09d-a84d-403a-8a16-be9996ce716d None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:32.497562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ad09d-a84d-403a-8a16-be9996ce716d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:32.497562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ad09d-a84d-403a-8a16-be9996ce716d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:32.514110 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-493ad09d-a84d-403a-8a16-be9996ce716d None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:32.515874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-493ad09d-a84d-403a-8a16-be9996ce716d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:32.524238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1035/2059] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:32.528624 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c56a5-8d0a-4507-9c95-43707715728f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:32.539246 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c56a5-8d0a-4507-9c95-43707715728f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:32.544103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33aae5a-572a-4d53-8878-8ceec1945c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:32.544733 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aae5a-572a-4d53-8878-8ceec1945c51 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:32.545013 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aae5a-572a-4d53-8878-8ceec1945c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:32.545287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aae5a-572a-4d53-8878-8ceec1945c51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:32.545970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33aae5a-572a-4d53-8878-8ceec1945c51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:07:32.546304 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33aae5a-572a-4d53-8878-8ceec1945c51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:32.547970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33aae5a-572a-4d53-8878-8ceec1945c51 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:32.562815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1025/2060] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:07:32.567877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ad09d-a84d-403a-8a16-be9996ce716d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:32.579980 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ad09d-a84d-403a-8a16-be9996ce716d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:32.585044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b9fe9-bccc-45d4-834b-ae5d942ac5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:32.585837 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b9fe9-bccc-45d4-834b-ae5d942ac5ea admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:32.586185 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b9fe9-bccc-45d4-834b-ae5d942ac5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:32.586553 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b9fe9-bccc-45d4-834b-ae5d942ac5ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:32.587127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b9fe9-bccc-45d4-834b-ae5d942ac5ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:32.589088 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b9fe9-bccc-45d4-834b-ae5d942ac5ea admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:32.713437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1036/2061] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:32 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:32.717197 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33aae5a-572a-4d53-8878-8ceec1945c51 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:32.727578 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33aae5a-572a-4d53-8878-8ceec1945c51 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:32.788584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bba754-6bac-4e80-9698-b2c35a7b7a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:32.789322 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bba754-6bac-4e80-9698-b2c35a7b7a9e admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:32.789610 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bba754-6bac-4e80-9698-b2c35a7b7a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:32.789912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bba754-6bac-4e80-9698-b2c35a7b7a9e admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:32.790577 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bba754-6bac-4e80-9698-b2c35a7b7a9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:32.792616 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bba754-6bac-4e80-9698-b2c35a7b7a9e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:33.017937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1026/2062] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:32 2022] POST /identity/v3/users => generated 286 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:07:33.026362 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b9fe9-bccc-45d4-834b-ae5d942ac5ea admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:33.040656 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b9fe9-bccc-45d4-834b-ae5d942ac5ea admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:33.050129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100a900-64b0-46ef-b117-409dae275ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:33.051761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100a900-64b0-46ef-b117-409dae275ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:33.051761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100a900-64b0-46ef-b117-409dae275ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:33.051761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100a900-64b0-46ef-b117-409dae275ff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:33.053910 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100a900-64b0-46ef-b117-409dae275ff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:33.054220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100a900-64b0-46ef-b117-409dae275ff5 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:33.081162 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1037/2063] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:33.082418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bba754-6bac-4e80-9698-b2c35a7b7a9e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:33.094688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bba754-6bac-4e80-9698-b2c35a7b7a9e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:33.100030 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8066d9-0146-418f-bde6-cdcf3ca32782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:33.100650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8066d9-0146-418f-bde6-cdcf3ca32782 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:33.100942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8066d9-0146-418f-bde6-cdcf3ca32782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:33.101296 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8066d9-0146-418f-bde6-cdcf3ca32782 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:33.101995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8066d9-0146-418f-bde6-cdcf3ca32782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:33.103705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8066d9-0146-418f-bde6-cdcf3ca32782 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:33.125363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1027/2064] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:33.129929 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100a900-64b0-46ef-b117-409dae275ff5 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:33.145632 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100a900-64b0-46ef-b117-409dae275ff5 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:33.161397 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6eee25-011c-4f32-8353-8b40a5e2359f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:33.162129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6eee25-011c-4f32-8353-8b40a5e2359f admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:33.162420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6eee25-011c-4f32-8353-8b40a5e2359f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:33.162748 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6eee25-011c-4f32-8353-8b40a5e2359f admin admin] PATH_INFO: `/v3/projects/6c9979087f9746b59bd1c2b4e0dbcd4b/users/820db9f8b3f2443a932f844758bba703/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:33.196697 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6eee25-011c-4f32-8353-8b40a5e2359f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c9979087f9746b59bd1c2b4e0dbcd4b, user_id=820db9f8b3f2443a932f844758bba703, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:33.199649 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6eee25-011c-4f32-8353-8b40a5e2359f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:33.343142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1038/2065] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:33 2022] PUT /identity/v3/projects/6c9979087f9746b59bd1c2b4e0dbcd4b/users/820db9f8b3f2443a932f844758bba703/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:33.350646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af7fb4-fe4f-41ff-bc42-41a3b8654dec None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:33.350646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af7fb4-fe4f-41ff-bc42-41a3b8654dec None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:33.351504 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af7fb4-fe4f-41ff-bc42-41a3b8654dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:33.372675 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b1af7fb4-fe4f-41ff-bc42-41a3b8654dec None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:33.374059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b1af7fb4-fe4f-41ff-bc42-41a3b8654dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:33.453469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1028/2066] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:33.466462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6eee25-011c-4f32-8353-8b40a5e2359f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:33.469021 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cb6eee25-011c-4f32-8353-8b40a5e2359f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:33.589033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:33.602185 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6eee25-011c-4f32-8353-8b40a5e2359f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:33.606543 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcf95f-8ae9-4328-92b5-7c89f32ac949 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:33.607679 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcf95f-8ae9-4328-92b5-7c89f32ac949 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:33.607966 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcf95f-8ae9-4328-92b5-7c89f32ac949 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:33.608283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcf95f-8ae9-4328-92b5-7c89f32ac949 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:33.620444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcf95f-8ae9-4328-92b5-7c89f32ac949 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:33.622233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcf95f-8ae9-4328-92b5-7c89f32ac949 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:33.634963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1039/2067] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:07:35.122310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b36fca-41cd-41c6-97f4-1a276b899ddb None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:35.122955 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b36fca-41cd-41c6-97f4-1a276b899ddb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:35.122955 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b36fca-41cd-41c6-97f4-1a276b899ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:35.156516 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-76b36fca-41cd-41c6-97f4-1a276b899ddb None None] MFA Rules not processed for user `820db9f8b3f2443a932f844758bba703`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:35.157979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-76b36fca-41cd-41c6-97f4-1a276b899ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:35.207282 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:35.256851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1029/2068] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:35.258702 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcf95f-8ae9-4328-92b5-7c89f32ac949 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:35.274164 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcf95f-8ae9-4328-92b5-7c89f32ac949 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:35.299378 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace6323-386e-468f-874f-004fd470005a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:35.300006 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace6323-386e-468f-874f-004fd470005a admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:35.300425 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace6323-386e-468f-874f-004fd470005a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:35.300805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace6323-386e-468f-874f-004fd470005a admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:35.301506 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace6323-386e-468f-874f-004fd470005a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:35.303389 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace6323-386e-468f-874f-004fd470005a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:35.420953 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1040/2069] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:35 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:07:35.427223 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b36fca-41cd-41c6-97f4-1a276b899ddb None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:35.444129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b36fca-41cd-41c6-97f4-1a276b899ddb None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:35.543294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b194f7-2440-4064-9b78-52649e2f248d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:35.543921 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b194f7-2440-4064-9b78-52649e2f248d admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:35.544202 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b194f7-2440-4064-9b78-52649e2f248d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:35.544481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b194f7-2440-4064-9b78-52649e2f248d admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:35.545128 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b194f7-2440-4064-9b78-52649e2f248d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:35.547126 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b194f7-2440-4064-9b78-52649e2f248d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:35.670526 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1030/2070] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:35 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:35.680485 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace6323-386e-468f-874f-004fd470005a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:35.692715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace6323-386e-468f-874f-004fd470005a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:35.697429 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59998f70-d27e-4226-bf63-7a991d8e9a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:35.698251 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59998f70-d27e-4226-bf63-7a991d8e9a1a admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:35.698694 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59998f70-d27e-4226-bf63-7a991d8e9a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:35.698972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59998f70-d27e-4226-bf63-7a991d8e9a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:35.699626 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59998f70-d27e-4226-bf63-7a991d8e9a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:35.701322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59998f70-d27e-4226-bf63-7a991d8e9a1a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:35.717645 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1041/2071] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:35.719935 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b194f7-2440-4064-9b78-52649e2f248d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:35.734155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b194f7-2440-4064-9b78-52649e2f248d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:35.739485 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac358e-c8a9-4a8e-b764-2b2502cdb0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:35.740195 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac358e-c8a9-4a8e-b764-2b2502cdb0a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:35.740483 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac358e-c8a9-4a8e-b764-2b2502cdb0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:35.740762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac358e-c8a9-4a8e-b764-2b2502cdb0a5 admin admin] PATH_INFO: `/v3/projects/205377aab8864eed98123c393006b6e0/users/c2c62b6ec2b74a548ead44ae26afa2a5/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:35.762531 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac358e-c8a9-4a8e-b764-2b2502cdb0a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=205377aab8864eed98123c393006b6e0, user_id=c2c62b6ec2b74a548ead44ae26afa2a5, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:35.765892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac358e-c8a9-4a8e-b764-2b2502cdb0a5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:35.894590 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1031/2072] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:35 2022] PUT /identity/v3/projects/205377aab8864eed98123c393006b6e0/users/c2c62b6ec2b74a548ead44ae26afa2a5/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:35.898982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59998f70-d27e-4226-bf63-7a991d8e9a1a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:35.910029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59998f70-d27e-4226-bf63-7a991d8e9a1a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:35.990150 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05e629-6a7a-4b67-b36c-6693cf2cd44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:36.000017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05e629-6a7a-4b67-b36c-6693cf2cd44a admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:36.000017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05e629-6a7a-4b67-b36c-6693cf2cd44a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:36.000017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05e629-6a7a-4b67-b36c-6693cf2cd44a admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:36.000017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05e629-6a7a-4b67-b36c-6693cf2cd44a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:36.000017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05e629-6a7a-4b67-b36c-6693cf2cd44a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:36.008643 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1042/2073] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:36.009977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac358e-c8a9-4a8e-b764-2b2502cdb0a5 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:36.022471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac358e-c8a9-4a8e-b764-2b2502cdb0a5 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:36.027773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9960fe-f863-4b32-ae9a-4dea61ca8a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:36.034164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9960fe-f863-4b32-ae9a-4dea61ca8a95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:36.034164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9960fe-f863-4b32-ae9a-4dea61ca8a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:36.034164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9960fe-f863-4b32-ae9a-4dea61ca8a95 admin admin] PATH_INFO: `/v3/projects/205377aab8864eed98123c393006b6e0/users/c2c62b6ec2b74a548ead44ae26afa2a5/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:36.034164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9960fe-f863-4b32-ae9a-4dea61ca8a95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=205377aab8864eed98123c393006b6e0, user_id=c2c62b6ec2b74a548ead44ae26afa2a5, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:36.034700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9960fe-f863-4b32-ae9a-4dea61ca8a95 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:36.048500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1032/2074] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:36 2022] PUT /identity/v3/projects/205377aab8864eed98123c393006b6e0/users/c2c62b6ec2b74a548ead44ae26afa2a5/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:38.378367 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3335710-9b1f-4fc7-9b72-fb7ca4b8f28f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:38.378923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3335710-9b1f-4fc7-9b72-fb7ca4b8f28f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:38.378923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3335710-9b1f-4fc7-9b72-fb7ca4b8f28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:38.412452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e3335710-9b1f-4fc7-9b72-fb7ca4b8f28f None None] MFA Rules not processed for user `c2c62b6ec2b74a548ead44ae26afa2a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:38.414022 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e3335710-9b1f-4fc7-9b72-fb7ca4b8f28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:38.509012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1043/2075] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:38.540378 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9960fe-f863-4b32-ae9a-4dea61ca8a95 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:38.546634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4f9960fe-f863-4b32-ae9a-4dea61ca8a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:38.613196 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9960fe-f863-4b32-ae9a-4dea61ca8a95 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:38.616885 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5546f-ac85-4bb4-8b5d-e8b3de77fa2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:38.617500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5546f-ac85-4bb4-8b5d-e8b3de77fa2c service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:38.617789 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5546f-ac85-4bb4-8b5d-e8b3de77fa2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:38.618097 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5546f-ac85-4bb4-8b5d-e8b3de77fa2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:38.672810 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:38.694334 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5546f-ac85-4bb4-8b5d-e8b3de77fa2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:38.703633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5546f-ac85-4bb4-8b5d-e8b3de77fa2c service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:38.708652 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1033/2076] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:38.909461 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3335710-9b1f-4fc7-9b72-fb7ca4b8f28f None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:38.910875 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e3335710-9b1f-4fc7-9b72-fb7ca4b8f28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:38.967909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:39.006602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3335710-9b1f-4fc7-9b72-fb7ca4b8f28f None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:39.011907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca27ffb-1c43-4cb5-ac9c-44658ad2e79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:39.012651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca27ffb-1c43-4cb5-ac9c-44658ad2e79b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:39.012946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca27ffb-1c43-4cb5-ac9c-44658ad2e79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:39.013245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca27ffb-1c43-4cb5-ac9c-44658ad2e79b admin admin] PATH_INFO: `/v3/users/6297f7c6455c4a0e884453e4c422034a` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:39.015420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca27ffb-1c43-4cb5-ac9c-44658ad2e79b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6297f7c6455c4a0e884453e4c422034a)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:39.017714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca27ffb-1c43-4cb5-ac9c-44658ad2e79b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:39.614233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9ca27ffb-1c43-4cb5-ac9c-44658ad2e79b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6297f7c6455c4a0e884453e4c422034a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:39.630911 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9ca27ffb-1c43-4cb5-ac9c-44658ad2e79b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6297f7c6455c4a0e884453e4c422034a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:39.643514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9ca27ffb-1c43-4cb5-ac9c-44658ad2e79b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6297f7c6455c4a0e884453e4c422034a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:39.915645 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9ca27ffb-1c43-4cb5-ac9c-44658ad2e79b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6297f7c6455c4a0e884453e4c422034a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:39.917344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9ca27ffb-1c43-4cb5-ac9c-44658ad2e79b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6297f7c6455c4a0e884453e4c422034a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:39.926454 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1044/2077] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:38 2022] DELETE /identity/v3/users/6297f7c6455c4a0e884453e4c422034a => generated 0 bytes in 1018 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:39.931243 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6b145-139c-4d6e-9a58-e5b32003f5a5 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:39.938013 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6b145-139c-4d6e-9a58-e5b32003f5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:39.938013 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6b145-139c-4d6e-9a58-e5b32003f5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:39.954848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e9f6b145-139c-4d6e-9a58-e5b32003f5a5 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:39.956125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e9f6b145-139c-4d6e-9a58-e5b32003f5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:40.049258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1034/2078] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:40.060170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca27ffb-1c43-4cb5-ac9c-44658ad2e79b admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:40.061752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9ca27ffb-1c43-4cb5-ac9c-44658ad2e79b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:40.186274 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca27ffb-1c43-4cb5-ac9c-44658ad2e79b admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:40.190731 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e3e22-7202-48d4-a3ff-1e5fa0dfe851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:40.191621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e3e22-7202-48d4-a3ff-1e5fa0dfe851 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:40.192588 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e3e22-7202-48d4-a3ff-1e5fa0dfe851 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:40.193500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e3e22-7202-48d4-a3ff-1e5fa0dfe851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:40.207105 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e3e22-7202-48d4-a3ff-1e5fa0dfe851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:40.209028 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e3e22-7202-48d4-a3ff-1e5fa0dfe851 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:40.224185 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1045/2079] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:40.385127 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6b145-139c-4d6e-9a58-e5b32003f5a5 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:40.387814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e9f6b145-139c-4d6e-9a58-e5b32003f5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:40.460450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6b145-139c-4d6e-9a58-e5b32003f5a5 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:40.465518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f789c-815d-4b79-9a47-916f46cf7764 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:40.465518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f789c-815d-4b79-9a47-916f46cf7764 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:40.466022 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f789c-815d-4b79-9a47-916f46cf7764 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:40.466375 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f789c-815d-4b79-9a47-916f46cf7764 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:40.468818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e46f789c-815d-4b79-9a47-916f46cf7764 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:40.574430 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f789c-815d-4b79-9a47-916f46cf7764 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:40.576774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f789c-815d-4b79-9a47-916f46cf7764 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:40.605812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1035/2080] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:40.826828 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e3e22-7202-48d4-a3ff-1e5fa0dfe851 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:40.833196 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fc5e3e22-7202-48d4-a3ff-1e5fa0dfe851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:40.852165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e3e22-7202-48d4-a3ff-1e5fa0dfe851 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:40.857431 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91adedb-8c23-4c1c-a245-bd88df6eddbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:40.858212 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91adedb-8c23-4c1c-a245-bd88df6eddbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:40.859014 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91adedb-8c23-4c1c-a245-bd88df6eddbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:40.859271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91adedb-8c23-4c1c-a245-bd88df6eddbd admin admin] PATH_INFO: `/v3/projects/7c5e7090d0c8426eb82d152a49029509` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:40.860517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91adedb-8c23-4c1c-a245-bd88df6eddbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c5e7090d0c8426eb82d152a49029509)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:40.862519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91adedb-8c23-4c1c-a245-bd88df6eddbd admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:41.005368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a91adedb-8c23-4c1c-a245-bd88df6eddbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c5e7090d0c8426eb82d152a49029509', 'request_id': 'req-a91adedb-8c23-4c1c-a245-bd88df6eddbd'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:41.007180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:41.097072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a91adedb-8c23-4c1c-a245-bd88df6eddbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c5e7090d0c8426eb82d152a49029509', 'request_id': 'req-a91adedb-8c23-4c1c-a245-bd88df6eddbd'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:41.104142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a91adedb-8c23-4c1c-a245-bd88df6eddbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c5e7090d0c8426eb82d152a49029509', 'request_id': 'req-a91adedb-8c23-4c1c-a245-bd88df6eddbd'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:41.114654 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a91adedb-8c23-4c1c-a245-bd88df6eddbd admin admin] The token cache is being invalidate because project 7c5e7090d0c8426eb82d152a49029509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:41.114928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a91adedb-8c23-4c1c-a245-bd88df6eddbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:41.115578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a91adedb-8c23-4c1c-a245-bd88df6eddbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:41.117073 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1046/2081] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:40 2022] DELETE /identity/v3/projects/7c5e7090d0c8426eb82d152a49029509 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:41.121334 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f789c-815d-4b79-9a47-916f46cf7764 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:41.133425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f789c-815d-4b79-9a47-916f46cf7764 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:41.177492 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:41.200070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda799ba-9b3b-438f-80db-abc9dc33909e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:41.200586 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda799ba-9b3b-438f-80db-abc9dc33909e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:41.201983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda799ba-9b3b-438f-80db-abc9dc33909e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:41.202346 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda799ba-9b3b-438f-80db-abc9dc33909e admin admin] PATH_INFO: `/v3/users/09aa8b184d8f4f52947d7fea43914f0e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:41.204095 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda799ba-9b3b-438f-80db-abc9dc33909e admin admin] RBAC: Authorizing `identity:delete_user(user_id=09aa8b184d8f4f52947d7fea43914f0e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:41.206156 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda799ba-9b3b-438f-80db-abc9dc33909e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:41.369471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fda799ba-9b3b-438f-80db-abc9dc33909e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09aa8b184d8f4f52947d7fea43914f0e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:41.382455 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fda799ba-9b3b-438f-80db-abc9dc33909e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09aa8b184d8f4f52947d7fea43914f0e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:41.389310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fda799ba-9b3b-438f-80db-abc9dc33909e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09aa8b184d8f4f52947d7fea43914f0e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:41.457638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fda799ba-9b3b-438f-80db-abc9dc33909e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09aa8b184d8f4f52947d7fea43914f0e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:41.458158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fda799ba-9b3b-438f-80db-abc9dc33909e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09aa8b184d8f4f52947d7fea43914f0e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:41.464848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1036/2082] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:41 2022] DELETE /identity/v3/users/09aa8b184d8f4f52947d7fea43914f0e => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:41.917341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91adedb-8c23-4c1c-a245-bd88df6eddbd admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:41.919415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a91adedb-8c23-4c1c-a245-bd88df6eddbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:41.942536 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91adedb-8c23-4c1c-a245-bd88df6eddbd admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:42.030194 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585a966-76aa-4d51-afdb-c890c8708c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:42.030870 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585a966-76aa-4d51-afdb-c890c8708c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:42.031177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585a966-76aa-4d51-afdb-c890c8708c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:42.031626 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585a966-76aa-4d51-afdb-c890c8708c50 admin admin] PATH_INFO: `/v3/projects/e2386a781f3d44ee92376b6d606de2dd` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:42.032747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585a966-76aa-4d51-afdb-c890c8708c50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2386a781f3d44ee92376b6d606de2dd)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:42.034781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585a966-76aa-4d51-afdb-c890c8708c50 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:42.183447 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d585a966-76aa-4d51-afdb-c890c8708c50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2386a781f3d44ee92376b6d606de2dd', 'request_id': 'req-d585a966-76aa-4d51-afdb-c890c8708c50'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:42.299285 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda799ba-9b3b-438f-80db-abc9dc33909e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:42.300902 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fda799ba-9b3b-438f-80db-abc9dc33909e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:42.303756 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d585a966-76aa-4d51-afdb-c890c8708c50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2386a781f3d44ee92376b6d606de2dd', 'request_id': 'req-d585a966-76aa-4d51-afdb-c890c8708c50'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:42.316759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d585a966-76aa-4d51-afdb-c890c8708c50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2386a781f3d44ee92376b6d606de2dd', 'request_id': 'req-d585a966-76aa-4d51-afdb-c890c8708c50'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:42.330629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d585a966-76aa-4d51-afdb-c890c8708c50 admin admin] The token cache is being invalidate because project e2386a781f3d44ee92376b6d606de2dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:42.330932 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d585a966-76aa-4d51-afdb-c890c8708c50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:42.331605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d585a966-76aa-4d51-afdb-c890c8708c50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:42.333176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1047/2083] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:41 2022] DELETE /identity/v3/projects/e2386a781f3d44ee92376b6d606de2dd => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:42.339224 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e24b8-7256-48e0-9635-0d61ece84217 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:42.340080 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e24b8-7256-48e0-9635-0d61ece84217 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:42.340371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e24b8-7256-48e0-9635-0d61ece84217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:42.365891 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-5c6e24b8-7256-48e0-9635-0d61ece84217 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:42.367306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5c6e24b8-7256-48e0-9635-0d61ece84217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:42.379560 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda799ba-9b3b-438f-80db-abc9dc33909e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:42.432113 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a8767-2faf-47b0-b94a-f16a637a112c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:42.432641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a8767-2faf-47b0-b94a-f16a637a112c service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:42.432753 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1048/2084] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:42.433000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a8767-2faf-47b0-b94a-f16a637a112c service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:42.433283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a8767-2faf-47b0-b94a-f16a637a112c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:42.435204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d25a8767-2faf-47b0-b94a-f16a637a112c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:42.441092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372c05c-bfe3-4498-8afa-ff3b12e45859 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:42.441407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372c05c-bfe3-4498-8afa-ff3b12e45859 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:42.441690 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372c05c-bfe3-4498-8afa-ff3b12e45859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:42.463605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6372c05c-bfe3-4498-8afa-ff3b12e45859 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:42.464855 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6372c05c-bfe3-4498-8afa-ff3b12e45859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:42.478779 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1049/2085] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:42.481605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:42.482543 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372c05c-bfe3-4498-8afa-ff3b12e45859 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:42.487127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:42.492956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372c05c-bfe3-4498-8afa-ff3b12e45859 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:42.498069 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6100ba-0fc2-4394-89f5-0da4b8b4691e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:42.499694 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6100ba-0fc2-4394-89f5-0da4b8b4691e admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:42.500186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6100ba-0fc2-4394-89f5-0da4b8b4691e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:42.500673 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6100ba-0fc2-4394-89f5-0da4b8b4691e admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:42.501627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6100ba-0fc2-4394-89f5-0da4b8b4691e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:07:42.502096 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6100ba-0fc2-4394-89f5-0da4b8b4691e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:42.504580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6100ba-0fc2-4394-89f5-0da4b8b4691e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:42.522442 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1050/2086] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:07:42.524454 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a8767-2faf-47b0-b94a-f16a637a112c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:42.526894 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a8767-2faf-47b0-b94a-f16a637a112c service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:42.527743 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6100ba-0fc2-4394-89f5-0da4b8b4691e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:42.542007 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1037/2087] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:42.546440 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6100ba-0fc2-4394-89f5-0da4b8b4691e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:42.555874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f8c39-bf17-48fb-ae03-a719553d922f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:42.556794 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f8c39-bf17-48fb-ae03-a719553d922f admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:42.557231 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f8c39-bf17-48fb-ae03-a719553d922f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:42.557846 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f8c39-bf17-48fb-ae03-a719553d922f admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:42.559053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f8c39-bf17-48fb-ae03-a719553d922f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:42.563150 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f8c39-bf17-48fb-ae03-a719553d922f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:42.699337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1051/2088] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:42 2022] POST /identity/v3/projects => generated 367 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:07:42.704291 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a8767-2faf-47b0-b94a-f16a637a112c service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:42.715321 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a8767-2faf-47b0-b94a-f16a637a112c service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:42.791099 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa1486-cec9-4afe-8659-3a98e8f6b1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:42.791847 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa1486-cec9-4afe-8659-3a98e8f6b1bf admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:42.792202 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa1486-cec9-4afe-8659-3a98e8f6b1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:42.792559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa1486-cec9-4afe-8659-3a98e8f6b1bf admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:42.793364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa1486-cec9-4afe-8659-3a98e8f6b1bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:42.808504 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa1486-cec9-4afe-8659-3a98e8f6b1bf admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:42.907456 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1038/2089] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:42 2022] POST /identity/v3/users => generated 284 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:42.912568 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f8c39-bf17-48fb-ae03-a719553d922f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:42.924379 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f8c39-bf17-48fb-ae03-a719553d922f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:42.928913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cd91e-a492-4725-b3dd-3420a35b9683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:42.929511 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cd91e-a492-4725-b3dd-3420a35b9683 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:42.929801 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cd91e-a492-4725-b3dd-3420a35b9683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:42.930105 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cd91e-a492-4725-b3dd-3420a35b9683 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:42.930743 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cd91e-a492-4725-b3dd-3420a35b9683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:42.932316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cd91e-a492-4725-b3dd-3420a35b9683 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:42.943137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1052/2090] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:42.949276 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa1486-cec9-4afe-8659-3a98e8f6b1bf admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:42.959971 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa1486-cec9-4afe-8659-3a98e8f6b1bf admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:42.964939 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a217d-e2dc-4c80-a23c-85efafc1d09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:42.965549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a217d-e2dc-4c80-a23c-85efafc1d09a admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:42.965994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a217d-e2dc-4c80-a23c-85efafc1d09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:42.966179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a217d-e2dc-4c80-a23c-85efafc1d09a admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:42.966835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a217d-e2dc-4c80-a23c-85efafc1d09a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:42.968562 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a217d-e2dc-4c80-a23c-85efafc1d09a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:42.979049 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1039/2091] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:42.985002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cd91e-a492-4725-b3dd-3420a35b9683 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:42.996024 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cd91e-a492-4725-b3dd-3420a35b9683 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:43.001440 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23440a4a-78eb-4927-b0ae-59887705ce25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:43.002141 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23440a4a-78eb-4927-b0ae-59887705ce25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:43.002422 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23440a4a-78eb-4927-b0ae-59887705ce25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:43.002700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23440a4a-78eb-4927-b0ae-59887705ce25 admin admin] PATH_INFO: `/v3/projects/7000d40e742e4e019dac35a2a69987a4/users/81775516e7b24074891cc7e808ff0d27/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:43.021152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23440a4a-78eb-4927-b0ae-59887705ce25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7000d40e742e4e019dac35a2a69987a4, user_id=81775516e7b24074891cc7e808ff0d27, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:43.024490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23440a4a-78eb-4927-b0ae-59887705ce25 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:43.120984 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1053/2092] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:42 2022] PUT /identity/v3/projects/7000d40e742e4e019dac35a2a69987a4/users/81775516e7b24074891cc7e808ff0d27/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:43.131967 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e68e5f-c310-426a-a01a-f5c9611da8e2 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:43.132258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e68e5f-c310-426a-a01a-f5c9611da8e2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:43.132540 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e68e5f-c310-426a-a01a-f5c9611da8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:43.165534 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-93e68e5f-c310-426a-a01a-f5c9611da8e2 None None] MFA Rules not processed for user `81775516e7b24074891cc7e808ff0d27`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:43.166854 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-93e68e5f-c310-426a-a01a-f5c9611da8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:43.237640 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1040/2093] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:43.244606 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23440a4a-78eb-4927-b0ae-59887705ce25 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:43.309507 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23440a4a-78eb-4927-b0ae-59887705ce25 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:43.314768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c2aec-fdf6-4925-bf9b-7c006ccfacaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:43.315520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c2aec-fdf6-4925-bf9b-7c006ccfacaa admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:43.315622 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c2aec-fdf6-4925-bf9b-7c006ccfacaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:43.315879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c2aec-fdf6-4925-bf9b-7c006ccfacaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:43.317076 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c2aec-fdf6-4925-bf9b-7c006ccfacaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:43.318454 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c2aec-fdf6-4925-bf9b-7c006ccfacaa admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:43.402545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1054/2094] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:43 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:43.406830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e68e5f-c310-426a-a01a-f5c9611da8e2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:43.418351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e68e5f-c310-426a-a01a-f5c9611da8e2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:43.454144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:43.480553 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ee581-a559-4d5a-95fd-5434fc007542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:43.481172 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ee581-a559-4d5a-95fd-5434fc007542 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:43.481454 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ee581-a559-4d5a-95fd-5434fc007542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:43.481757 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ee581-a559-4d5a-95fd-5434fc007542 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:43.482411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ee581-a559-4d5a-95fd-5434fc007542 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:43.484494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ee581-a559-4d5a-95fd-5434fc007542 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:43.617588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1041/2095] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:43 2022] POST /identity/v3/users => generated 290 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:43.618846 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c2aec-fdf6-4925-bf9b-7c006ccfacaa admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:43.630220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c2aec-fdf6-4925-bf9b-7c006ccfacaa admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:43.636530 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6cdfc-e7ee-4f14-b239-4ee8954381f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:43.637248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6cdfc-e7ee-4f14-b239-4ee8954381f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:43.637627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6cdfc-e7ee-4f14-b239-4ee8954381f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:43.638483 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6cdfc-e7ee-4f14-b239-4ee8954381f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:43.638848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f6cdfc-e7ee-4f14-b239-4ee8954381f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:43.643174 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f6cdfc-e7ee-4f14-b239-4ee8954381f1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:43.659773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1055/2096] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:43.661690 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ee581-a559-4d5a-95fd-5434fc007542 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:43.673609 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ee581-a559-4d5a-95fd-5434fc007542 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:43.678677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea67dd-dbcc-4eeb-87ef-fb2aff638e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:43.679480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea67dd-dbcc-4eeb-87ef-fb2aff638e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:43.679768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea67dd-dbcc-4eeb-87ef-fb2aff638e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:43.680053 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea67dd-dbcc-4eeb-87ef-fb2aff638e7e admin admin] PATH_INFO: `/v3/projects/6259f8bd9d6a4611a70f0ca816040a0f/users/494d6cb7e4df4710b31258944ce76944/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:43.700707 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea67dd-dbcc-4eeb-87ef-fb2aff638e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6259f8bd9d6a4611a70f0ca816040a0f, user_id=494d6cb7e4df4710b31258944ce76944, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:43.703755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea67dd-dbcc-4eeb-87ef-fb2aff638e7e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:43.771846 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1042/2097] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:43 2022] PUT /identity/v3/projects/6259f8bd9d6a4611a70f0ca816040a0f/users/494d6cb7e4df4710b31258944ce76944/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:43.776479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6cdfc-e7ee-4f14-b239-4ee8954381f1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:43.788499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6cdfc-e7ee-4f14-b239-4ee8954381f1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:43.851312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837028a-1f2e-40e6-b605-ce62757bd771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:43.852359 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837028a-1f2e-40e6-b605-ce62757bd771 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:43.852999 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837028a-1f2e-40e6-b605-ce62757bd771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:43.853410 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837028a-1f2e-40e6-b605-ce62757bd771 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:43.854359 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837028a-1f2e-40e6-b605-ce62757bd771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:43.859447 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837028a-1f2e-40e6-b605-ce62757bd771 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:43.873784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1056/2098] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:43.877674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea67dd-dbcc-4eeb-87ef-fb2aff638e7e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:43.894698 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea67dd-dbcc-4eeb-87ef-fb2aff638e7e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:43.899895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8c892-5f81-4f30-a3fe-cb90a782c8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:43.900660 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8c892-5f81-4f30-a3fe-cb90a782c8d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:43.901017 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8c892-5f81-4f30-a3fe-cb90a782c8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:43.901316 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8c892-5f81-4f30-a3fe-cb90a782c8d3 admin admin] PATH_INFO: `/v3/projects/6259f8bd9d6a4611a70f0ca816040a0f/users/494d6cb7e4df4710b31258944ce76944/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:43.903374 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8c892-5f81-4f30-a3fe-cb90a782c8d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6259f8bd9d6a4611a70f0ca816040a0f, user_id=494d6cb7e4df4710b31258944ce76944, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:43.907267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8c892-5f81-4f30-a3fe-cb90a782c8d3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:43.919953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1043/2099] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:43 2022] PUT /identity/v3/projects/6259f8bd9d6a4611a70f0ca816040a0f/users/494d6cb7e4df4710b31258944ce76944/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:07:43.934008 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31319370-6d80-42d5-9623-82d420156887 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:43.934008 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31319370-6d80-42d5-9623-82d420156887 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:43.934008 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31319370-6d80-42d5-9623-82d420156887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:43.942811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:43.972682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-31319370-6d80-42d5-9623-82d420156887 None None] MFA Rules not processed for user `494d6cb7e4df4710b31258944ce76944`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:43.973949 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-31319370-6d80-42d5-9623-82d420156887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:44.053144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1057/2100] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:44.073691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8c892-5f81-4f30-a3fe-cb90a782c8d3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:44.075698 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-24c8c892-5f81-4f30-a3fe-cb90a782c8d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:44.140271 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8c892-5f81-4f30-a3fe-cb90a782c8d3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:44.144117 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9f385-d48d-4ac8-b878-0cd9233859cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:44.144737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9f385-d48d-4ac8-b878-0cd9233859cc service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:44.145018 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9f385-d48d-4ac8-b878-0cd9233859cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:44.145293 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9f385-d48d-4ac8-b878-0cd9233859cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:44.157275 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9f385-d48d-4ac8-b878-0cd9233859cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:44.158771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9f385-d48d-4ac8-b878-0cd9233859cc service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:44.169445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1044/2101] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:44.944591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31319370-6d80-42d5-9623-82d420156887 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:44.965725 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31319370-6d80-42d5-9623-82d420156887 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:44.971257 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2127a3-9cc8-4c8b-94fa-54e843303821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:44.972263 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2127a3-9cc8-4c8b-94fa-54e843303821 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:44.972741 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2127a3-9cc8-4c8b-94fa-54e843303821 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:44.973197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2127a3-9cc8-4c8b-94fa-54e843303821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:45.053371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2127a3-9cc8-4c8b-94fa-54e843303821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:45.054877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2127a3-9cc8-4c8b-94fa-54e843303821 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:45.068088 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1058/2102] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:45.550422 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9f385-d48d-4ac8-b878-0cd9233859cc service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:45.551942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-12b9f385-d48d-4ac8-b878-0cd9233859cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:45.667930 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9f385-d48d-4ac8-b878-0cd9233859cc service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:45.674097 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f88b62-9d67-47ac-af6a-36e34607303c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:45.674960 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f88b62-9d67-47ac-af6a-36e34607303c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:45.675427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f88b62-9d67-47ac-af6a-36e34607303c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:45.675790 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f88b62-9d67-47ac-af6a-36e34607303c admin admin] PATH_INFO: `/v3/users/220c19e1cbfd4d0eb9602f8390117ee7` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:45.677300 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f88b62-9d67-47ac-af6a-36e34607303c admin admin] RBAC: Authorizing `identity:delete_user(user_id=220c19e1cbfd4d0eb9602f8390117ee7)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:45.679785 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f88b62-9d67-47ac-af6a-36e34607303c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:45.929547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:45.951374 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-86f88b62-9d67-47ac-af6a-36e34607303c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '220c19e1cbfd4d0eb9602f8390117ee7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:45.967301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-86f88b62-9d67-47ac-af6a-36e34607303c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '220c19e1cbfd4d0eb9602f8390117ee7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:45.975689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-86f88b62-9d67-47ac-af6a-36e34607303c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '220c19e1cbfd4d0eb9602f8390117ee7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:46.052182 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-86f88b62-9d67-47ac-af6a-36e34607303c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '220c19e1cbfd4d0eb9602f8390117ee7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:46.053273 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-86f88b62-9d67-47ac-af6a-36e34607303c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '220c19e1cbfd4d0eb9602f8390117ee7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:46.060006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1045/2103] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:45 2022] DELETE /identity/v3/users/220c19e1cbfd4d0eb9602f8390117ee7 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:46.065379 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f58a7-a092-4011-a2c8-06c7b7a07dfc None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:46.065924 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f58a7-a092-4011-a2c8-06c7b7a07dfc None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:46.066020 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f58a7-a092-4011-a2c8-06c7b7a07dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:46.092237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f66f58a7-a092-4011-a2c8-06c7b7a07dfc None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:46.094232 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f66f58a7-a092-4011-a2c8-06c7b7a07dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:46.165150 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1059/2104] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:46.175580 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f88b62-9d67-47ac-af6a-36e34607303c admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:46.180822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-86f88b62-9d67-47ac-af6a-36e34607303c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:46.240495 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f88b62-9d67-47ac-af6a-36e34607303c admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:46.244459 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddaf89-d86f-4a0e-9681-8031c430a259 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:46.245156 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddaf89-d86f-4a0e-9681-8031c430a259 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:46.245570 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddaf89-d86f-4a0e-9681-8031c430a259 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:46.245899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddaf89-d86f-4a0e-9681-8031c430a259 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:46.256872 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ddaf89-d86f-4a0e-9681-8031c430a259 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:46.258384 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ddaf89-d86f-4a0e-9681-8031c430a259 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:46.271171 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1046/2105] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:46.762725 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f58a7-a092-4011-a2c8-06c7b7a07dfc None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:46.764254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f66f58a7-a092-4011-a2c8-06c7b7a07dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:46.779753 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f58a7-a092-4011-a2c8-06c7b7a07dfc None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:46.784748 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cdd371-0623-4927-b399-58517b8bed61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:46.785518 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cdd371-0623-4927-b399-58517b8bed61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:46.789840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cdd371-0623-4927-b399-58517b8bed61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:46.789840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cdd371-0623-4927-b399-58517b8bed61 admin admin] PATH_INFO: `/v3/projects/bef4d06eb14b4a2482dd3fea2339f40d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:46.789840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cdd371-0623-4927-b399-58517b8bed61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bef4d06eb14b4a2482dd3fea2339f40d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:46.789840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cdd371-0623-4927-b399-58517b8bed61 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:46.824975 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:46.939287 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-19cdd371-0623-4927-b399-58517b8bed61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bef4d06eb14b4a2482dd3fea2339f40d', 'request_id': 'req-19cdd371-0623-4927-b399-58517b8bed61'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:47.046511 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-19cdd371-0623-4927-b399-58517b8bed61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bef4d06eb14b4a2482dd3fea2339f40d', 'request_id': 'req-19cdd371-0623-4927-b399-58517b8bed61'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:47.054119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-19cdd371-0623-4927-b399-58517b8bed61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bef4d06eb14b4a2482dd3fea2339f40d', 'request_id': 'req-19cdd371-0623-4927-b399-58517b8bed61'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:47.074193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-19cdd371-0623-4927-b399-58517b8bed61 admin admin] The token cache is being invalidate because project bef4d06eb14b4a2482dd3fea2339f40d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:47.074193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-19cdd371-0623-4927-b399-58517b8bed61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:47.074193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-19cdd371-0623-4927-b399-58517b8bed61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:47.074193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1060/2106] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:46 2022] DELETE /identity/v3/projects/bef4d06eb14b4a2482dd3fea2339f40d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:47.078061 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddaf89-d86f-4a0e-9681-8031c430a259 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:47.088882 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddaf89-d86f-4a0e-9681-8031c430a259 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:47.155569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb3952-31e6-4b1d-abca-425807f16027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:47.156222 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb3952-31e6-4b1d-abca-425807f16027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:47.156509 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb3952-31e6-4b1d-abca-425807f16027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:47.156789 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb3952-31e6-4b1d-abca-425807f16027 admin admin] PATH_INFO: `/v3/users/ae8f3f1d99b644c78e3a3cb6637ba43b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:47.157967 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb3952-31e6-4b1d-abca-425807f16027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae8f3f1d99b644c78e3a3cb6637ba43b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:47.159986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb3952-31e6-4b1d-abca-425807f16027 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:47.364955 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7efb3952-31e6-4b1d-abca-425807f16027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae8f3f1d99b644c78e3a3cb6637ba43b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:47.372027 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:47.384774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7efb3952-31e6-4b1d-abca-425807f16027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae8f3f1d99b644c78e3a3cb6637ba43b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:47.392419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7efb3952-31e6-4b1d-abca-425807f16027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae8f3f1d99b644c78e3a3cb6637ba43b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:47.497376 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7efb3952-31e6-4b1d-abca-425807f16027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae8f3f1d99b644c78e3a3cb6637ba43b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:47.498058 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7efb3952-31e6-4b1d-abca-425807f16027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae8f3f1d99b644c78e3a3cb6637ba43b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:47.505618 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1047/2107] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:47 2022] DELETE /identity/v3/users/ae8f3f1d99b644c78e3a3cb6637ba43b => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:47.981598 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cdd371-0623-4927-b399-58517b8bed61 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:47.985046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-19cdd371-0623-4927-b399-58517b8bed61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:48.008053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cdd371-0623-4927-b399-58517b8bed61 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:48.088087 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aabd53-5e23-438f-ad18-0456079159ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:48.089001 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aabd53-5e23-438f-ad18-0456079159ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:48.089001 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aabd53-5e23-438f-ad18-0456079159ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:48.089246 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aabd53-5e23-438f-ad18-0456079159ee admin admin] PATH_INFO: `/v3/projects/a983b75ccf294ffea19d55b14ee2a015` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:48.090375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aabd53-5e23-438f-ad18-0456079159ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=a983b75ccf294ffea19d55b14ee2a015)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:48.092280 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aabd53-5e23-438f-ad18-0456079159ee admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:48.239510 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-42aabd53-5e23-438f-ad18-0456079159ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a983b75ccf294ffea19d55b14ee2a015', 'request_id': 'req-42aabd53-5e23-438f-ad18-0456079159ee'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:48.336288 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-42aabd53-5e23-438f-ad18-0456079159ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a983b75ccf294ffea19d55b14ee2a015', 'request_id': 'req-42aabd53-5e23-438f-ad18-0456079159ee'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:48.347505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-42aabd53-5e23-438f-ad18-0456079159ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a983b75ccf294ffea19d55b14ee2a015', 'request_id': 'req-42aabd53-5e23-438f-ad18-0456079159ee'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:48.366126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-42aabd53-5e23-438f-ad18-0456079159ee admin admin] The token cache is being invalidate because project a983b75ccf294ffea19d55b14ee2a015 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:48.366428 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-42aabd53-5e23-438f-ad18-0456079159ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:48.367294 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-42aabd53-5e23-438f-ad18-0456079159ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:48.370907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1061/2108] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:47 2022] DELETE /identity/v3/projects/a983b75ccf294ffea19d55b14ee2a015 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:48.378153 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb3952-31e6-4b1d-abca-425807f16027 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:48.379606 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7efb3952-31e6-4b1d-abca-425807f16027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:48.395768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb3952-31e6-4b1d-abca-425807f16027 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:48.461274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc40e4-33bf-4747-a175-afd57724697b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:48.461848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc40e4-33bf-4747-a175-afd57724697b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:48.462170 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc40e4-33bf-4747-a175-afd57724697b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:48.462451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc40e4-33bf-4747-a175-afd57724697b admin admin] PATH_INFO: `/v3/users/eb4b5843d2e746c2b3c8513a3921a18c` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:48.463626 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc40e4-33bf-4747-a175-afd57724697b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb4b5843d2e746c2b3c8513a3921a18c)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:48.465563 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc40e4-33bf-4747-a175-afd57724697b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:48.649099 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-42fc40e4-33bf-4747-a175-afd57724697b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb4b5843d2e746c2b3c8513a3921a18c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:48.669354 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-42fc40e4-33bf-4747-a175-afd57724697b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb4b5843d2e746c2b3c8513a3921a18c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:48.679360 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-42fc40e4-33bf-4747-a175-afd57724697b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb4b5843d2e746c2b3c8513a3921a18c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:48.754531 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-42fc40e4-33bf-4747-a175-afd57724697b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb4b5843d2e746c2b3c8513a3921a18c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:48.755095 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-42fc40e4-33bf-4747-a175-afd57724697b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb4b5843d2e746c2b3c8513a3921a18c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:48.762420 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1048/2109] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:48 2022] DELETE /identity/v3/users/eb4b5843d2e746c2b3c8513a3921a18c => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:49.245462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aabd53-5e23-438f-ad18-0456079159ee admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:49.247109 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-42aabd53-5e23-438f-ad18-0456079159ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:49.274836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aabd53-5e23-438f-ad18-0456079159ee admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:49.330058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:49.351698 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2880b9-05ca-42ad-be8f-952a708b90f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:49.352381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2880b9-05ca-42ad-be8f-952a708b90f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:49.352666 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2880b9-05ca-42ad-be8f-952a708b90f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:49.352944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2880b9-05ca-42ad-be8f-952a708b90f3 admin admin] PATH_INFO: `/v3/projects/1d7ecf8648244d9d9c8237c47775fc7e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:49.354140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2880b9-05ca-42ad-be8f-952a708b90f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d7ecf8648244d9d9c8237c47775fc7e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:49.356051 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2880b9-05ca-42ad-be8f-952a708b90f3 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:49.482882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3e2880b9-05ca-42ad-be8f-952a708b90f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d7ecf8648244d9d9c8237c47775fc7e', 'request_id': 'req-3e2880b9-05ca-42ad-be8f-952a708b90f3'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:49.554942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3e2880b9-05ca-42ad-be8f-952a708b90f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d7ecf8648244d9d9c8237c47775fc7e', 'request_id': 'req-3e2880b9-05ca-42ad-be8f-952a708b90f3'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:49.563222 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3e2880b9-05ca-42ad-be8f-952a708b90f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d7ecf8648244d9d9c8237c47775fc7e', 'request_id': 'req-3e2880b9-05ca-42ad-be8f-952a708b90f3'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:49.576472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3e2880b9-05ca-42ad-be8f-952a708b90f3 admin admin] The token cache is being invalidate because project 1d7ecf8648244d9d9c8237c47775fc7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:49.576790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3e2880b9-05ca-42ad-be8f-952a708b90f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:49.577500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3e2880b9-05ca-42ad-be8f-952a708b90f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:49.579312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1062/2110] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:49 2022] DELETE /identity/v3/projects/1d7ecf8648244d9d9c8237c47775fc7e => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:49.585391 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e459ff-c2ba-42fb-b136-837ee7c827bb None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:49.585914 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e459ff-c2ba-42fb-b136-837ee7c827bb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:49.586135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e459ff-c2ba-42fb-b136-837ee7c827bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:49.611973 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c4e459ff-c2ba-42fb-b136-837ee7c827bb None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:49.613210 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c4e459ff-c2ba-42fb-b136-837ee7c827bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:49.662086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:49.677765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1049/2111] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:49.686527 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882ba15-942a-46f9-97d8-16d52aa6c2d4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:49.687327 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882ba15-942a-46f9-97d8-16d52aa6c2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:49.687612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882ba15-942a-46f9-97d8-16d52aa6c2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:49.709929 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0882ba15-942a-46f9-97d8-16d52aa6c2d4 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:49.713053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0882ba15-942a-46f9-97d8-16d52aa6c2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:49.724547 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1063/2112] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:49.729771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e459ff-c2ba-42fb-b136-837ee7c827bb None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:49.746194 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e459ff-c2ba-42fb-b136-837ee7c827bb None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:49.758422 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6998f-9697-47f9-93be-ebf97a15be36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:49.759698 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6998f-9697-47f9-93be-ebf97a15be36 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:49.759984 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6998f-9697-47f9-93be-ebf97a15be36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:49.760337 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6998f-9697-47f9-93be-ebf97a15be36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:49.761092 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6998f-9697-47f9-93be-ebf97a15be36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:07:49.761538 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6998f-9697-47f9-93be-ebf97a15be36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:49.764495 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6998f-9697-47f9-93be-ebf97a15be36 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:49.792686 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1050/2113] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:07:49.800592 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882ba15-942a-46f9-97d8-16d52aa6c2d4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:49.813796 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882ba15-942a-46f9-97d8-16d52aa6c2d4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:49.838442 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46e8bf-62e9-4c53-ae67-6e25ca886ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:49.839199 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46e8bf-62e9-4c53-ae67-6e25ca886ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:49.839535 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46e8bf-62e9-4c53-ae67-6e25ca886ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:49.839874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46e8bf-62e9-4c53-ae67-6e25ca886ec6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:49.840590 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46e8bf-62e9-4c53-ae67-6e25ca886ec6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:49.842629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46e8bf-62e9-4c53-ae67-6e25ca886ec6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:49.985536 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1064/2114] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:49 2022] POST /identity/v3/projects => generated 393 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:49.990086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6998f-9697-47f9-93be-ebf97a15be36 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:50.000656 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6998f-9697-47f9-93be-ebf97a15be36 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:50.066529 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fb1f0-0111-4aae-af51-a5b06dc34580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:50.067179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fb1f0-0111-4aae-af51-a5b06dc34580 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:50.067466 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fb1f0-0111-4aae-af51-a5b06dc34580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:50.067766 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fb1f0-0111-4aae-af51-a5b06dc34580 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:50.068427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787fb1f0-0111-4aae-af51-a5b06dc34580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:50.070496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787fb1f0-0111-4aae-af51-a5b06dc34580 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:50.165602 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1051/2115] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:49 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:50.168246 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46e8bf-62e9-4c53-ae67-6e25ca886ec6 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:50.181280 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46e8bf-62e9-4c53-ae67-6e25ca886ec6 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:50.186233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39697830-576f-4a88-9fc9-5caea43a4e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:50.186868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39697830-576f-4a88-9fc9-5caea43a4e53 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:50.187180 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39697830-576f-4a88-9fc9-5caea43a4e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:50.187456 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39697830-576f-4a88-9fc9-5caea43a4e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:50.188102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39697830-576f-4a88-9fc9-5caea43a4e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:50.189703 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39697830-576f-4a88-9fc9-5caea43a4e53 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:50.201804 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1065/2116] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:50.211173 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fb1f0-0111-4aae-af51-a5b06dc34580 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:50.222333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fb1f0-0111-4aae-af51-a5b06dc34580 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:50.227095 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a0407-3bd5-42c2-93ab-be9d272ab99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:50.227704 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a0407-3bd5-42c2-93ab-be9d272ab99f admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:50.227977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a0407-3bd5-42c2-93ab-be9d272ab99f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:50.228278 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a0407-3bd5-42c2-93ab-be9d272ab99f admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:50.229742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a0407-3bd5-42c2-93ab-be9d272ab99f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:50.230781 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a0407-3bd5-42c2-93ab-be9d272ab99f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:50.242928 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1052/2117] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:07:50.247326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39697830-576f-4a88-9fc9-5caea43a4e53 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:50.258731 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39697830-576f-4a88-9fc9-5caea43a4e53 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:50.264212 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff732a4-8b20-4a48-997c-5c3dfabc6dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:50.265439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff732a4-8b20-4a48-997c-5c3dfabc6dbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:50.265725 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff732a4-8b20-4a48-997c-5c3dfabc6dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:50.266048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff732a4-8b20-4a48-997c-5c3dfabc6dbf admin admin] PATH_INFO: `/v3/projects/f86c6b8f38c844df97d670fc6be13b60/users/518565b0b31847da9290e73bcfd35d5d/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:50.290855 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff732a4-8b20-4a48-997c-5c3dfabc6dbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f86c6b8f38c844df97d670fc6be13b60, user_id=518565b0b31847da9290e73bcfd35d5d, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:50.293753 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff732a4-8b20-4a48-997c-5c3dfabc6dbf admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:50.375275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1066/2118] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:50 2022] PUT /identity/v3/projects/f86c6b8f38c844df97d670fc6be13b60/users/518565b0b31847da9290e73bcfd35d5d/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:50.386495 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094a868-2d4f-4787-b913-6a69ea587aa8 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:50.386783 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094a868-2d4f-4787-b913-6a69ea587aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:50.387059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094a868-2d4f-4787-b913-6a69ea587aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:50.418172 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f094a868-2d4f-4787-b913-6a69ea587aa8 None None] MFA Rules not processed for user `518565b0b31847da9290e73bcfd35d5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:50.419423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f094a868-2d4f-4787-b913-6a69ea587aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:50.497667 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1053/2119] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:50.506702 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff732a4-8b20-4a48-997c-5c3dfabc6dbf admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:50.585763 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff732a4-8b20-4a48-997c-5c3dfabc6dbf admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:50.590486 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f175d-1756-40c4-a5b1-ff2c2b4372fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:50.591138 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f175d-1756-40c4-a5b1-ff2c2b4372fc admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:50.591792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f175d-1756-40c4-a5b1-ff2c2b4372fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:50.591792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f175d-1756-40c4-a5b1-ff2c2b4372fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:50.592311 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f175d-1756-40c4-a5b1-ff2c2b4372fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:50.594441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f175d-1756-40c4-a5b1-ff2c2b4372fc admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:50.606298 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:50.684255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1067/2120] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:50 2022] POST /identity/v3/projects => generated 391 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:07:50.688265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094a868-2d4f-4787-b913-6a69ea587aa8 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:50.702044 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094a868-2d4f-4787-b913-6a69ea587aa8 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:50.761050 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:50.777560 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8ed1f-0fd1-4bc7-88fc-6f213f6312e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:50.778336 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8ed1f-0fd1-4bc7-88fc-6f213f6312e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:50.778701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8ed1f-0fd1-4bc7-88fc-6f213f6312e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:50.779052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8ed1f-0fd1-4bc7-88fc-6f213f6312e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:50.779930 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8ed1f-0fd1-4bc7-88fc-6f213f6312e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:50.782564 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8ed1f-0fd1-4bc7-88fc-6f213f6312e4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:50.870422 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1054/2121] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:50 2022] POST /identity/v3/users => generated 302 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:50.874396 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f175d-1756-40c4-a5b1-ff2c2b4372fc admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:50.887238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f175d-1756-40c4-a5b1-ff2c2b4372fc admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:50.894645 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a23e48-03f9-4177-a48d-e807c5a3252b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:50.899878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a23e48-03f9-4177-a48d-e807c5a3252b admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:50.899878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a23e48-03f9-4177-a48d-e807c5a3252b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:50.899878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a23e48-03f9-4177-a48d-e807c5a3252b admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:50.899878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a23e48-03f9-4177-a48d-e807c5a3252b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:50.899878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a23e48-03f9-4177-a48d-e807c5a3252b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:50.912965 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1068/2122] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:50.914206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8ed1f-0fd1-4bc7-88fc-6f213f6312e4 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:50.930965 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8ed1f-0fd1-4bc7-88fc-6f213f6312e4 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:50.940898 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b023a2-8834-4528-aee9-c70bff6bebb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:50.941694 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b023a2-8834-4528-aee9-c70bff6bebb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:50.942088 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b023a2-8834-4528-aee9-c70bff6bebb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:50.942447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b023a2-8834-4528-aee9-c70bff6bebb9 admin admin] PATH_INFO: `/v3/projects/2b1f002e95844f86a432acc1cda8840d/users/f257210e7b8c45f2a4839678f0e429aa/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:50.963139 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b023a2-8834-4528-aee9-c70bff6bebb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b1f002e95844f86a432acc1cda8840d, user_id=f257210e7b8c45f2a4839678f0e429aa, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:50.967253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b023a2-8834-4528-aee9-c70bff6bebb9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:51.071445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1055/2123] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:50 2022] PUT /identity/v3/projects/2b1f002e95844f86a432acc1cda8840d/users/f257210e7b8c45f2a4839678f0e429aa/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:51.076147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a23e48-03f9-4177-a48d-e807c5a3252b admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:51.089248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a23e48-03f9-4177-a48d-e807c5a3252b admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:51.181202 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e13fbe-f424-4ecb-a991-c1ee3ec19cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:51.182283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e13fbe-f424-4ecb-a991-c1ee3ec19cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:51.182575 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e13fbe-f424-4ecb-a991-c1ee3ec19cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:51.182858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e13fbe-f424-4ecb-a991-c1ee3ec19cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:51.183599 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e13fbe-f424-4ecb-a991-c1ee3ec19cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:51.185756 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e13fbe-f424-4ecb-a991-c1ee3ec19cc2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:51.199095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1069/2124] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:07:51.203433 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b023a2-8834-4528-aee9-c70bff6bebb9 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:51.214834 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b023a2-8834-4528-aee9-c70bff6bebb9 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:51.219724 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c253d-3b5c-44eb-8dcc-6bf5ef2f9423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:51.220391 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c253d-3b5c-44eb-8dcc-6bf5ef2f9423 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:51.220683 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c253d-3b5c-44eb-8dcc-6bf5ef2f9423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:51.220969 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c253d-3b5c-44eb-8dcc-6bf5ef2f9423 admin admin] PATH_INFO: `/v3/projects/2b1f002e95844f86a432acc1cda8840d/users/f257210e7b8c45f2a4839678f0e429aa/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:51.223513 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c253d-3b5c-44eb-8dcc-6bf5ef2f9423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b1f002e95844f86a432acc1cda8840d, user_id=f257210e7b8c45f2a4839678f0e429aa, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:51.226698 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c253d-3b5c-44eb-8dcc-6bf5ef2f9423 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:51.247274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1056/2125] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:51 2022] PUT /identity/v3/projects/2b1f002e95844f86a432acc1cda8840d/users/f257210e7b8c45f2a4839678f0e429aa/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:51.251615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673365b-5b7f-4b30-9936-e53c8b682922 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:51.252132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673365b-5b7f-4b30-9936-e53c8b682922 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:51.252323 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673365b-5b7f-4b30-9936-e53c8b682922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:51.285163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e673365b-5b7f-4b30-9936-e53c8b682922 None None] MFA Rules not processed for user `f257210e7b8c45f2a4839678f0e429aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:51.286433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e673365b-5b7f-4b30-9936-e53c8b682922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:51.389129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1070/2126] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:51 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:51.401085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c253d-3b5c-44eb-8dcc-6bf5ef2f9423 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:51.402953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-309c253d-3b5c-44eb-8dcc-6bf5ef2f9423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:51.474388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c253d-3b5c-44eb-8dcc-6bf5ef2f9423 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:51.479019 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065cfde-26ac-47fb-9315-fc577472d630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:51.479767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065cfde-26ac-47fb-9315-fc577472d630 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:51.480258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065cfde-26ac-47fb-9315-fc577472d630 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:51.480402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065cfde-26ac-47fb-9315-fc577472d630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:51.492989 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065cfde-26ac-47fb-9315-fc577472d630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:51.494587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065cfde-26ac-47fb-9315-fc577472d630 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:51.507078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1057/2127] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:51 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:51.623826 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673365b-5b7f-4b30-9936-e53c8b682922 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:51.651208 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673365b-5b7f-4b30-9936-e53c8b682922 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:51.658248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125616a-4f38-4d72-b59e-b23559470a14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:51.659252 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125616a-4f38-4d72-b59e-b23559470a14 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:51.659874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125616a-4f38-4d72-b59e-b23559470a14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:51.660269 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125616a-4f38-4d72-b59e-b23559470a14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:51.761161 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4125616a-4f38-4d72-b59e-b23559470a14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:51.763697 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4125616a-4f38-4d72-b59e-b23559470a14 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:51.773536 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:51.780875 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1071/2128] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:53.446231 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065cfde-26ac-47fb-9315-fc577472d630 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:53.447717 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3065cfde-26ac-47fb-9315-fc577472d630 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:53.515902 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065cfde-26ac-47fb-9315-fc577472d630 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:53.520449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e0ba6-c456-4686-ba63-70502caacc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:53.521076 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e0ba6-c456-4686-ba63-70502caacc3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:53.521358 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e0ba6-c456-4686-ba63-70502caacc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:53.521629 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e0ba6-c456-4686-ba63-70502caacc3c admin admin] PATH_INFO: `/v3/users/81775516e7b24074891cc7e808ff0d27` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:53.522766 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e0ba6-c456-4686-ba63-70502caacc3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=81775516e7b24074891cc7e808ff0d27)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:53.524954 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e0ba6-c456-4686-ba63-70502caacc3c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:53.791384 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:53.794876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e9e0ba6-c456-4686-ba63-70502caacc3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81775516e7b24074891cc7e808ff0d27'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:53.810545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e9e0ba6-c456-4686-ba63-70502caacc3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81775516e7b24074891cc7e808ff0d27'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:53.817547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e9e0ba6-c456-4686-ba63-70502caacc3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81775516e7b24074891cc7e808ff0d27'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:53.898104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e9e0ba6-c456-4686-ba63-70502caacc3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81775516e7b24074891cc7e808ff0d27'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:53.899079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e9e0ba6-c456-4686-ba63-70502caacc3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81775516e7b24074891cc7e808ff0d27'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:53.906304 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1058/2129] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:53 2022] DELETE /identity/v3/users/81775516e7b24074891cc7e808ff0d27 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:53.910845 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b2185-a5c3-481f-9230-83e7acf4fd39 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:53.911370 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b2185-a5c3-481f-9230-83e7acf4fd39 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:53.911462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b2185-a5c3-481f-9230-83e7acf4fd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:53.932706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a58b2185-a5c3-481f-9230-83e7acf4fd39 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:53.933971 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a58b2185-a5c3-481f-9230-83e7acf4fd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:54.033789 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1072/2130] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:54.045476 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e0ba6-c456-4686-ba63-70502caacc3c admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:54.050236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1e9e0ba6-c456-4686-ba63-70502caacc3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:54.116261 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e0ba6-c456-4686-ba63-70502caacc3c admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:54.120472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d455c0-67d1-401e-a9e3-2c63fb155553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:54.121320 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d455c0-67d1-401e-a9e3-2c63fb155553 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:54.121683 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d455c0-67d1-401e-a9e3-2c63fb155553 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:54.122063 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d455c0-67d1-401e-a9e3-2c63fb155553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:54.140812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d455c0-67d1-401e-a9e3-2c63fb155553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:54.142889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d455c0-67d1-401e-a9e3-2c63fb155553 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:54.171558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1059/2131] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:54.677195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b2185-a5c3-481f-9230-83e7acf4fd39 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:54.678708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a58b2185-a5c3-481f-9230-83e7acf4fd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:54.694110 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b2185-a5c3-481f-9230-83e7acf4fd39 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:54.698925 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6cc19-3730-48a7-bb9b-a34fdefe4ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:54.699563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6cc19-3730-48a7-bb9b-a34fdefe4ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:54.699845 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6cc19-3730-48a7-bb9b-a34fdefe4ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:54.700409 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6cc19-3730-48a7-bb9b-a34fdefe4ed6 admin admin] PATH_INFO: `/v3/projects/7000d40e742e4e019dac35a2a69987a4` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:54.701310 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6cc19-3730-48a7-bb9b-a34fdefe4ed6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7000d40e742e4e019dac35a2a69987a4)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:54.703873 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6cc19-3730-48a7-bb9b-a34fdefe4ed6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:54.876103 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b9b6cc19-3730-48a7-bb9b-a34fdefe4ed6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7000d40e742e4e019dac35a2a69987a4', 'request_id': 'req-b9b6cc19-3730-48a7-bb9b-a34fdefe4ed6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:54.972776 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b9b6cc19-3730-48a7-bb9b-a34fdefe4ed6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7000d40e742e4e019dac35a2a69987a4', 'request_id': 'req-b9b6cc19-3730-48a7-bb9b-a34fdefe4ed6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:54.983676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b9b6cc19-3730-48a7-bb9b-a34fdefe4ed6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7000d40e742e4e019dac35a2a69987a4', 'request_id': 'req-b9b6cc19-3730-48a7-bb9b-a34fdefe4ed6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:55.007472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b9b6cc19-3730-48a7-bb9b-a34fdefe4ed6 admin admin] The token cache is being invalidate because project 7000d40e742e4e019dac35a2a69987a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:55.007772 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b9b6cc19-3730-48a7-bb9b-a34fdefe4ed6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:55.010059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b9b6cc19-3730-48a7-bb9b-a34fdefe4ed6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:55.010059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1073/2132] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:54 2022] DELETE /identity/v3/projects/7000d40e742e4e019dac35a2a69987a4 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:55.014147 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d455c0-67d1-401e-a9e3-2c63fb155553 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:55.025314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d455c0-67d1-401e-a9e3-2c63fb155553 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:55.119628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d428f6-84fc-4e32-a6b4-c0a60431e0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:55.119628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d428f6-84fc-4e32-a6b4-c0a60431e0ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:55.120187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d428f6-84fc-4e32-a6b4-c0a60431e0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:55.120613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d428f6-84fc-4e32-a6b4-c0a60431e0ff admin admin] PATH_INFO: `/v3/users/494d6cb7e4df4710b31258944ce76944` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:55.122982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d428f6-84fc-4e32-a6b4-c0a60431e0ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=494d6cb7e4df4710b31258944ce76944)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:55.125626 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d428f6-84fc-4e32-a6b4-c0a60431e0ff admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:55.315272 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d4d428f6-84fc-4e32-a6b4-c0a60431e0ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '494d6cb7e4df4710b31258944ce76944'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:55.358915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d4d428f6-84fc-4e32-a6b4-c0a60431e0ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '494d6cb7e4df4710b31258944ce76944'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:55.361354 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:55.373737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d4d428f6-84fc-4e32-a6b4-c0a60431e0ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '494d6cb7e4df4710b31258944ce76944'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:55.463234 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d4d428f6-84fc-4e32-a6b4-c0a60431e0ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '494d6cb7e4df4710b31258944ce76944'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:55.463899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d4d428f6-84fc-4e32-a6b4-c0a60431e0ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '494d6cb7e4df4710b31258944ce76944'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:55.470903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1060/2133] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:55 2022] DELETE /identity/v3/users/494d6cb7e4df4710b31258944ce76944 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:55.981981 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6cc19-3730-48a7-bb9b-a34fdefe4ed6 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:55.992722 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b9b6cc19-3730-48a7-bb9b-a34fdefe4ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:56.029560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6cc19-3730-48a7-bb9b-a34fdefe4ed6 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:56.118749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b8dcb-6f8b-4c5a-a26b-ebe402412bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:56.119493 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b8dcb-6f8b-4c5a-a26b-ebe402412bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:56.119786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b8dcb-6f8b-4c5a-a26b-ebe402412bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:56.120068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b8dcb-6f8b-4c5a-a26b-ebe402412bbd admin admin] PATH_INFO: `/v3/projects/6259f8bd9d6a4611a70f0ca816040a0f` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:56.121256 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484b8dcb-6f8b-4c5a-a26b-ebe402412bbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6259f8bd9d6a4611a70f0ca816040a0f)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:56.123451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484b8dcb-6f8b-4c5a-a26b-ebe402412bbd admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:56.289220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:56.327722 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-484b8dcb-6f8b-4c5a-a26b-ebe402412bbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6259f8bd9d6a4611a70f0ca816040a0f', 'request_id': 'req-484b8dcb-6f8b-4c5a-a26b-ebe402412bbd'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:56.440822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-484b8dcb-6f8b-4c5a-a26b-ebe402412bbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6259f8bd9d6a4611a70f0ca816040a0f', 'request_id': 'req-484b8dcb-6f8b-4c5a-a26b-ebe402412bbd'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:56.450847 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-484b8dcb-6f8b-4c5a-a26b-ebe402412bbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6259f8bd9d6a4611a70f0ca816040a0f', 'request_id': 'req-484b8dcb-6f8b-4c5a-a26b-ebe402412bbd'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:56.481712 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-484b8dcb-6f8b-4c5a-a26b-ebe402412bbd admin admin] The token cache is being invalidate because project 6259f8bd9d6a4611a70f0ca816040a0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:07:56.482013 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-484b8dcb-6f8b-4c5a-a26b-ebe402412bbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:56.484963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-484b8dcb-6f8b-4c5a-a26b-ebe402412bbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:07:56.489655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1074/2134] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:07:55 2022] DELETE /identity/v3/projects/6259f8bd9d6a4611a70f0ca816040a0f => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:56.498024 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f105733-c80b-494f-9b20-edddf00366be None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:56.498936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f105733-c80b-494f-9b20-edddf00366be None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:56.499760 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f105733-c80b-494f-9b20-edddf00366be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:56.543879 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8f105733-c80b-494f-9b20-edddf00366be None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:56.545383 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8f105733-c80b-494f-9b20-edddf00366be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:56.642510 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1061/2135] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:56.652360 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d595de-f6ba-4c7c-82d0-d12b108280e6 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:56.652852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d595de-f6ba-4c7c-82d0-d12b108280e6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:56.653150 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d595de-f6ba-4c7c-82d0-d12b108280e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:56.688599 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-c3d595de-f6ba-4c7c-82d0-d12b108280e6 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:56.690763 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c3d595de-f6ba-4c7c-82d0-d12b108280e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:56.703815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1075/2136] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:07:56.708608 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f105733-c80b-494f-9b20-edddf00366be None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:56.722381 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f105733-c80b-494f-9b20-edddf00366be None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:56.731812 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd5186-cf90-459d-92d5-15a90f421ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:56.733015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd5186-cf90-459d-92d5-15a90f421ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:56.733235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd5186-cf90-459d-92d5-15a90f421ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:56.734162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd5186-cf90-459d-92d5-15a90f421ea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:56.741040 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd5186-cf90-459d-92d5-15a90f421ea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:07:56.741040 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd5186-cf90-459d-92d5-15a90f421ea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:56.741040 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd5186-cf90-459d-92d5-15a90f421ea3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:56.751007 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1062/2137] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 12:07:56.757937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d595de-f6ba-4c7c-82d0-d12b108280e6 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:56.770615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d595de-f6ba-4c7c-82d0-d12b108280e6 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:56.777847 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd2455-44d1-4208-b4a8-b8501d193807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:56.778433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd2455-44d1-4208-b4a8-b8501d193807 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:56.778714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd2455-44d1-4208-b4a8-b8501d193807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:56.778992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd2455-44d1-4208-b4a8-b8501d193807 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:56.779637 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd2455-44d1-4208-b4a8-b8501d193807 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:56.782131 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd2455-44d1-4208-b4a8-b8501d193807 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:56.913814 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1076/2138] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:56 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:56.918590 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd5186-cf90-459d-92d5-15a90f421ea3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:56.932326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd5186-cf90-459d-92d5-15a90f421ea3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:56.998642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703f320-59a5-4afa-be96-f8a64606e667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:56.999366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703f320-59a5-4afa-be96-f8a64606e667 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:56.999640 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703f320-59a5-4afa-be96-f8a64606e667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:56.999916 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703f320-59a5-4afa-be96-f8a64606e667 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:57.000559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703f320-59a5-4afa-be96-f8a64606e667 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:57.002678 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703f320-59a5-4afa-be96-f8a64606e667 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:57.110978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1063/2139] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:56 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:07:57.115934 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd2455-44d1-4208-b4a8-b8501d193807 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:57.130704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd2455-44d1-4208-b4a8-b8501d193807 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:57.136893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ee90c-74fa-4c86-9e9f-bacd28782056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:57.137663 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ee90c-74fa-4c86-9e9f-bacd28782056 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:57.138154 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ee90c-74fa-4c86-9e9f-bacd28782056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:57.138540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ee90c-74fa-4c86-9e9f-bacd28782056 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:57.139296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ee90c-74fa-4c86-9e9f-bacd28782056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:57.141158 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ee90c-74fa-4c86-9e9f-bacd28782056 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:57.157377 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1077/2140] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:07:57.161538 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703f320-59a5-4afa-be96-f8a64606e667 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:57.172585 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703f320-59a5-4afa-be96-f8a64606e667 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:57.177478 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c361d-7669-4230-80ce-58273a1c9b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:57.178150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c361d-7669-4230-80ce-58273a1c9b91 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:57.178434 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c361d-7669-4230-80ce-58273a1c9b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:57.178711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c361d-7669-4230-80ce-58273a1c9b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:57.179360 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c361d-7669-4230-80ce-58273a1c9b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:57.180964 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c361d-7669-4230-80ce-58273a1c9b91 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:57.193770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1064/2141] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:07:57.198066 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ee90c-74fa-4c86-9e9f-bacd28782056 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:57.210816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ee90c-74fa-4c86-9e9f-bacd28782056 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:57.216256 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b14c8-c957-4f2b-8878-4cda8f0e8bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:57.217076 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b14c8-c957-4f2b-8878-4cda8f0e8bbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:57.217447 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b14c8-c957-4f2b-8878-4cda8f0e8bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:57.218056 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b14c8-c957-4f2b-8878-4cda8f0e8bbf admin admin] PATH_INFO: `/v3/projects/8ffaa44538614f07929f501c9dd71467/users/c6c900bc2401450ebcc43db2b0499faa/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:57.237455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b14c8-c957-4f2b-8878-4cda8f0e8bbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ffaa44538614f07929f501c9dd71467, user_id=c6c900bc2401450ebcc43db2b0499faa, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:57.240296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b14c8-c957-4f2b-8878-4cda8f0e8bbf admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:57.324428 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1078/2142] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:57 2022] PUT /identity/v3/projects/8ffaa44538614f07929f501c9dd71467/users/c6c900bc2401450ebcc43db2b0499faa/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:57.336410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20b28a-3f40-4d56-b228-1b4199967ba7 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:57.336727 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20b28a-3f40-4d56-b228-1b4199967ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:57.337006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20b28a-3f40-4d56-b228-1b4199967ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:57.369508 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ef20b28a-3f40-4d56-b228-1b4199967ba7 None None] MFA Rules not processed for user `c6c900bc2401450ebcc43db2b0499faa`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:57.370935 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ef20b28a-3f40-4d56-b228-1b4199967ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:57.419201 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:57.459189 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1065/2143] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:57.463435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b14c8-c957-4f2b-8878-4cda8f0e8bbf admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:57.606806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b14c8-c957-4f2b-8878-4cda8f0e8bbf admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:57.628016 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79082f9-d9db-4ff3-913a-119109d4c462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:57.629067 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79082f9-d9db-4ff3-913a-119109d4c462 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:57.630323 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79082f9-d9db-4ff3-913a-119109d4c462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:57.630323 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79082f9-d9db-4ff3-913a-119109d4c462 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:57.631513 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79082f9-d9db-4ff3-913a-119109d4c462 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:57.634836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79082f9-d9db-4ff3-913a-119109d4c462 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:57.755202 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1079/2144] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:07:57 2022] POST /identity/v3/projects => generated 383 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:07:57.761534 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20b28a-3f40-4d56-b228-1b4199967ba7 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:57.776458 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20b28a-3f40-4d56-b228-1b4199967ba7 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:57.858966 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6f886-5278-40d2-9b81-248f70949884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:57.859633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6f886-5278-40d2-9b81-248f70949884 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:57.860343 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6f886-5278-40d2-9b81-248f70949884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:57.860718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6f886-5278-40d2-9b81-248f70949884 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:57.861679 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6f886-5278-40d2-9b81-248f70949884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:57.863747 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6f886-5278-40d2-9b81-248f70949884 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:57.956327 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1066/2145] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:07:57 2022] POST /identity/v3/users => generated 298 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:07:57.960404 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79082f9-d9db-4ff3-913a-119109d4c462 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:57.976243 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79082f9-d9db-4ff3-913a-119109d4c462 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:57.983656 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34fa9c-c5c9-4777-ae3c-d0dcb62c28fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:57.984696 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34fa9c-c5c9-4777-ae3c-d0dcb62c28fb admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:57.985185 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34fa9c-c5c9-4777-ae3c-d0dcb62c28fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:57.985678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34fa9c-c5c9-4777-ae3c-d0dcb62c28fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:57.986870 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34fa9c-c5c9-4777-ae3c-d0dcb62c28fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:57.990183 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34fa9c-c5c9-4777-ae3c-d0dcb62c28fb admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:58.005669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1080/2146] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:07:58.010312 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6f886-5278-40d2-9b81-248f70949884 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:58.022348 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6f886-5278-40d2-9b81-248f70949884 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:58.027707 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba0aa5-6958-4eb3-9607-2622ead1ce16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:58.028588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba0aa5-6958-4eb3-9607-2622ead1ce16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:58.028774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba0aa5-6958-4eb3-9607-2622ead1ce16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:58.029056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba0aa5-6958-4eb3-9607-2622ead1ce16 admin admin] PATH_INFO: `/v3/projects/014fde5ff75449b9a9e6c8fe4630f4a9/users/8464dbd143a04815aa5019e244ee7e89/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:58.048436 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba0aa5-6958-4eb3-9607-2622ead1ce16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=014fde5ff75449b9a9e6c8fe4630f4a9, user_id=8464dbd143a04815aa5019e244ee7e89, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:58.051249 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba0aa5-6958-4eb3-9607-2622ead1ce16 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:58.136230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1067/2147] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:58 2022] PUT /identity/v3/projects/014fde5ff75449b9a9e6c8fe4630f4a9/users/8464dbd143a04815aa5019e244ee7e89/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:58.137456 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34fa9c-c5c9-4777-ae3c-d0dcb62c28fb admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:58.150018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34fa9c-c5c9-4777-ae3c-d0dcb62c28fb admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:58.178889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:58.225165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c847cdd-63fe-4943-87ec-b8df1d2f0975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:58.225761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c847cdd-63fe-4943-87ec-b8df1d2f0975 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:58.226268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c847cdd-63fe-4943-87ec-b8df1d2f0975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:58.226611 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c847cdd-63fe-4943-87ec-b8df1d2f0975 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:58.227396 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c847cdd-63fe-4943-87ec-b8df1d2f0975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:58.229333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c847cdd-63fe-4943-87ec-b8df1d2f0975 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:58.242373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1081/2148] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:07:58.246304 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba0aa5-6958-4eb3-9607-2622ead1ce16 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:58.258366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba0aa5-6958-4eb3-9607-2622ead1ce16 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:58.266143 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d77da2-cdd7-4f3c-9278-4bd127f2416b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:58.266955 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d77da2-cdd7-4f3c-9278-4bd127f2416b admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:58.267485 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d77da2-cdd7-4f3c-9278-4bd127f2416b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:58.267769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d77da2-cdd7-4f3c-9278-4bd127f2416b admin admin] PATH_INFO: `/v3/projects/014fde5ff75449b9a9e6c8fe4630f4a9/users/8464dbd143a04815aa5019e244ee7e89/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:58.269759 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d77da2-cdd7-4f3c-9278-4bd127f2416b admin admin] RBAC: Authorizing `identity:create_grant(project_id=014fde5ff75449b9a9e6c8fe4630f4a9, user_id=8464dbd143a04815aa5019e244ee7e89, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:58.272804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d77da2-cdd7-4f3c-9278-4bd127f2416b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:58.304198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1068/2149] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:07:58 2022] PUT /identity/v3/projects/014fde5ff75449b9a9e6c8fe4630f4a9/users/8464dbd143a04815aa5019e244ee7e89/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:07:58.314489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57c5d2-08cc-4c2f-9700-a27d0baeab09 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:58.315071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57c5d2-08cc-4c2f-9700-a27d0baeab09 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:58.315474 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57c5d2-08cc-4c2f-9700-a27d0baeab09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:58.355897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9f57c5d2-08cc-4c2f-9700-a27d0baeab09 None None] MFA Rules not processed for user `8464dbd143a04815aa5019e244ee7e89`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:07:58.357390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9f57c5d2-08cc-4c2f-9700-a27d0baeab09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:58.471885 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1082/2150] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:07:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:07:58.501196 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d77da2-cdd7-4f3c-9278-4bd127f2416b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:58.503807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-29d77da2-cdd7-4f3c-9278-4bd127f2416b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:58.595440 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d77da2-cdd7-4f3c-9278-4bd127f2416b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:58.599822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6843f59-8329-46b3-9c29-71f16f2250bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:58.600703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6843f59-8329-46b3-9c29-71f16f2250bc service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:58.601244 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6843f59-8329-46b3-9c29-71f16f2250bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:58.601827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6843f59-8329-46b3-9c29-71f16f2250bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:58.613500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6843f59-8329-46b3-9c29-71f16f2250bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:58.615185 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6843f59-8329-46b3-9c29-71f16f2250bc service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:58.629806 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1069/2151] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:58.860538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57c5d2-08cc-4c2f-9700-a27d0baeab09 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:58.871899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57c5d2-08cc-4c2f-9700-a27d0baeab09 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:58.875607 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee31781-4099-478f-bfcf-64cee86f5751 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:58.876317 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee31781-4099-478f-bfcf-64cee86f5751 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:58.876652 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee31781-4099-478f-bfcf-64cee86f5751 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:58.876991 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee31781-4099-478f-bfcf-64cee86f5751 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:58.957061 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee31781-4099-478f-bfcf-64cee86f5751 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:58.958724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee31781-4099-478f-bfcf-64cee86f5751 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:58.972992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1083/2152] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:07:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:07:59.534774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6843f59-8329-46b3-9c29-71f16f2250bc service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:59.536482 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c6843f59-8329-46b3-9c29-71f16f2250bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:07:59.600002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:59.615277 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee31781-4099-478f-bfcf-64cee86f5751 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:07:59.719453 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6843f59-8329-46b3-9c29-71f16f2250bc service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:59.728007 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2174361-d02d-48f8-a114-bcf747eabf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:59.728914 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2174361-d02d-48f8-a114-bcf747eabf13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:59.729378 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2174361-d02d-48f8-a114-bcf747eabf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:59.729957 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2174361-d02d-48f8-a114-bcf747eabf13 admin admin] PATH_INFO: `/v3/users/518565b0b31847da9290e73bcfd35d5d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:59.732346 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2174361-d02d-48f8-a114-bcf747eabf13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=518565b0b31847da9290e73bcfd35d5d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:59.734046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:07:59.737591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2174361-d02d-48f8-a114-bcf747eabf13 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:07:59.763628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee31781-4099-478f-bfcf-64cee86f5751 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:07:59.773477 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f278d06-7554-4ff4-a91c-cda347031326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:07:59.773477 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f278d06-7554-4ff4-a91c-cda347031326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:07:59.773477 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f278d06-7554-4ff4-a91c-cda347031326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:07:59.773477 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f278d06-7554-4ff4-a91c-cda347031326 admin admin] PATH_INFO: `/v3/users/c6c900bc2401450ebcc43db2b0499faa` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:07:59.776698 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f278d06-7554-4ff4-a91c-cda347031326 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6c900bc2401450ebcc43db2b0499faa)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:07:59.780420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f278d06-7554-4ff4-a91c-cda347031326 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:00.047027 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a2174361-d02d-48f8-a114-bcf747eabf13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '518565b0b31847da9290e73bcfd35d5d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:00.072996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a2174361-d02d-48f8-a114-bcf747eabf13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '518565b0b31847da9290e73bcfd35d5d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:00.074186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3f278d06-7554-4ff4-a91c-cda347031326 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6c900bc2401450ebcc43db2b0499faa'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:00.089108 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a2174361-d02d-48f8-a114-bcf747eabf13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '518565b0b31847da9290e73bcfd35d5d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:00.120385 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3f278d06-7554-4ff4-a91c-cda347031326 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6c900bc2401450ebcc43db2b0499faa'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:00.138422 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3f278d06-7554-4ff4-a91c-cda347031326 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6c900bc2401450ebcc43db2b0499faa'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:00.236547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a2174361-d02d-48f8-a114-bcf747eabf13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '518565b0b31847da9290e73bcfd35d5d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:00.237204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a2174361-d02d-48f8-a114-bcf747eabf13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '518565b0b31847da9290e73bcfd35d5d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:00.244298 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1070/2153] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:59 2022] DELETE /identity/v3/users/518565b0b31847da9290e73bcfd35d5d => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:00.250782 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc5aa0-2507-488f-be74-9c690d2285ff None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:00.251196 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc5aa0-2507-488f-be74-9c690d2285ff None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:00.251678 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc5aa0-2507-488f-be74-9c690d2285ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:00.285240 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a3bc5aa0-2507-488f-be74-9c690d2285ff None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:00.287212 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a3bc5aa0-2507-488f-be74-9c690d2285ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:00.333003 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3f278d06-7554-4ff4-a91c-cda347031326 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6c900bc2401450ebcc43db2b0499faa'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:00.333768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3f278d06-7554-4ff4-a91c-cda347031326 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6c900bc2401450ebcc43db2b0499faa'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:00.342304 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1084/2154] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:07:59 2022] DELETE /identity/v3/users/c6c900bc2401450ebcc43db2b0499faa => generated 0 bytes in 728 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:00.348576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808958d-4a3c-41df-af5b-b068d13a6cea None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:00.349077 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808958d-4a3c-41df-af5b-b068d13a6cea None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:00.349346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808958d-4a3c-41df-af5b-b068d13a6cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:00.398047 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-5808958d-4a3c-41df-af5b-b068d13a6cea None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:00.399741 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5808958d-4a3c-41df-af5b-b068d13a6cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:00.471738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1071/2155] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:00.485680 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc5aa0-2507-488f-be74-9c690d2285ff None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:00.496153 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a3bc5aa0-2507-488f-be74-9c690d2285ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:00.595924 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1085/2156] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:08:00.612320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808958d-4a3c-41df-af5b-b068d13a6cea None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:00.613811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5808958d-4a3c-41df-af5b-b068d13a6cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:00.653295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc5aa0-2507-488f-be74-9c690d2285ff None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:00.674701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:00.757771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808958d-4a3c-41df-af5b-b068d13a6cea None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:00.764914 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1238203-de3c-4783-80f7-953685c03250 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:00.766560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1238203-de3c-4783-80f7-953685c03250 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:00.770559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1238203-de3c-4783-80f7-953685c03250 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:00.770559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1238203-de3c-4783-80f7-953685c03250 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:00.782727 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999808f1-24bd-47cd-8ab0-15543b9764ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:00.784172 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999808f1-24bd-47cd-8ab0-15543b9764ee service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:00.784715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999808f1-24bd-47cd-8ab0-15543b9764ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:00.785236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999808f1-24bd-47cd-8ab0-15543b9764ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:00.791651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1238203-de3c-4783-80f7-953685c03250 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:00.795123 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1238203-de3c-4783-80f7-953685c03250 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:00.808248 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999808f1-24bd-47cd-8ab0-15543b9764ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:00.810854 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999808f1-24bd-47cd-8ab0-15543b9764ee service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:00.816677 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1086/2157] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:08:00.851249 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1072/2158] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:08:01.481215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1238203-de3c-4783-80f7-953685c03250 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:01.482464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999808f1-24bd-47cd-8ab0-15543b9764ee service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:01.484351 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b1238203-de3c-4783-80f7-953685c03250 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:01.484789 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-999808f1-24bd-47cd-8ab0-15543b9764ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:01.499056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999808f1-24bd-47cd-8ab0-15543b9764ee service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:01.505100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0926f-0768-4d01-9c3a-ed4418318343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:01.506037 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0926f-0768-4d01-9c3a-ed4418318343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:01.506327 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0926f-0768-4d01-9c3a-ed4418318343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:01.506616 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0926f-0768-4d01-9c3a-ed4418318343 admin admin] PATH_INFO: `/v3/projects/f86c6b8f38c844df97d670fc6be13b60` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:01.507772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0926f-0768-4d01-9c3a-ed4418318343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f86c6b8f38c844df97d670fc6be13b60)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:01.509754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0926f-0768-4d01-9c3a-ed4418318343 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:01.512720 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1238203-de3c-4783-80f7-953685c03250 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:01.523350 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76c20b-dc0a-412c-8a66-2288a76615ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:01.525415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76c20b-dc0a-412c-8a66-2288a76615ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:01.526054 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76c20b-dc0a-412c-8a66-2288a76615ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:01.526740 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76c20b-dc0a-412c-8a66-2288a76615ea admin admin] PATH_INFO: `/v3/projects/8ffaa44538614f07929f501c9dd71467` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:01.528919 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76c20b-dc0a-412c-8a66-2288a76615ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ffaa44538614f07929f501c9dd71467)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:01.532797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76c20b-dc0a-412c-8a66-2288a76615ea admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:01.535000 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:01.698076 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4e76c20b-dc0a-412c-8a66-2288a76615ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ffaa44538614f07929f501c9dd71467', 'request_id': 'req-4e76c20b-dc0a-412c-8a66-2288a76615ea'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:01.710979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1bd0926f-0768-4d01-9c3a-ed4418318343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f86c6b8f38c844df97d670fc6be13b60', 'request_id': 'req-1bd0926f-0768-4d01-9c3a-ed4418318343'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:01.800874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4e76c20b-dc0a-412c-8a66-2288a76615ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ffaa44538614f07929f501c9dd71467', 'request_id': 'req-4e76c20b-dc0a-412c-8a66-2288a76615ea'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:01.808188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4e76c20b-dc0a-412c-8a66-2288a76615ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ffaa44538614f07929f501c9dd71467', 'request_id': 'req-4e76c20b-dc0a-412c-8a66-2288a76615ea'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:01.820994 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4e76c20b-dc0a-412c-8a66-2288a76615ea admin admin] The token cache is being invalidate because project 8ffaa44538614f07929f501c9dd71467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:08:01.821533 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4e76c20b-dc0a-412c-8a66-2288a76615ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:01.822425 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4e76c20b-dc0a-412c-8a66-2288a76615ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:01.824642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1087/2159] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:08:01 2022] DELETE /identity/v3/projects/8ffaa44538614f07929f501c9dd71467 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:01.825531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76c20b-dc0a-412c-8a66-2288a76615ea admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:01.827557 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4e76c20b-dc0a-412c-8a66-2288a76615ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:01.885135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1bd0926f-0768-4d01-9c3a-ed4418318343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f86c6b8f38c844df97d670fc6be13b60', 'request_id': 'req-1bd0926f-0768-4d01-9c3a-ed4418318343'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:01.895833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1bd0926f-0768-4d01-9c3a-ed4418318343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f86c6b8f38c844df97d670fc6be13b60', 'request_id': 'req-1bd0926f-0768-4d01-9c3a-ed4418318343'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:01.916694 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1bd0926f-0768-4d01-9c3a-ed4418318343 admin admin] The token cache is being invalidate because project f86c6b8f38c844df97d670fc6be13b60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:08:01.917071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1bd0926f-0768-4d01-9c3a-ed4418318343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:01.918636 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1bd0926f-0768-4d01-9c3a-ed4418318343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:01.924744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1073/2160] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:08:01 2022] DELETE /identity/v3/projects/f86c6b8f38c844df97d670fc6be13b60 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:08:01.925108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76c20b-dc0a-412c-8a66-2288a76615ea admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:01.937177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0926f-0768-4d01-9c3a-ed4418318343 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:01.938877 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1bd0926f-0768-4d01-9c3a-ed4418318343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:02.028850 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0926f-0768-4d01-9c3a-ed4418318343 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:02.033780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aba74c-8971-4a48-83ab-f659d25366d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:02.034543 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aba74c-8971-4a48-83ab-f659d25366d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:02.034835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aba74c-8971-4a48-83ab-f659d25366d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:02.035114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aba74c-8971-4a48-83ab-f659d25366d2 admin admin] PATH_INFO: `/v3/users/8464dbd143a04815aa5019e244ee7e89` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:02.036321 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aba74c-8971-4a48-83ab-f659d25366d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8464dbd143a04815aa5019e244ee7e89)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:02.037792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59629ce5-996e-4bc1-89ae-6a750cea7171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:02.038339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aba74c-8971-4a48-83ab-f659d25366d2 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:02.038505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59629ce5-996e-4bc1-89ae-6a750cea7171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:02.038747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59629ce5-996e-4bc1-89ae-6a750cea7171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:02.039025 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59629ce5-996e-4bc1-89ae-6a750cea7171 admin admin] PATH_INFO: `/v3/users/5d6b6321705947b6a395f77c11b1c6a5` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:02.040256 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59629ce5-996e-4bc1-89ae-6a750cea7171 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d6b6321705947b6a395f77c11b1c6a5)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:02.042385 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59629ce5-996e-4bc1-89ae-6a750cea7171 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:02.345574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:02.354713 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-59629ce5-996e-4bc1-89ae-6a750cea7171 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d6b6321705947b6a395f77c11b1c6a5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:02.358975 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:02.372208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-f1aba74c-8971-4a48-83ab-f659d25366d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8464dbd143a04815aa5019e244ee7e89'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:02.384125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-59629ce5-996e-4bc1-89ae-6a750cea7171 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d6b6321705947b6a395f77c11b1c6a5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:02.390122 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-f1aba74c-8971-4a48-83ab-f659d25366d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8464dbd143a04815aa5019e244ee7e89'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:02.391788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-59629ce5-996e-4bc1-89ae-6a750cea7171 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d6b6321705947b6a395f77c11b1c6a5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:02.398130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-f1aba74c-8971-4a48-83ab-f659d25366d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8464dbd143a04815aa5019e244ee7e89'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:02.487761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-59629ce5-996e-4bc1-89ae-6a750cea7171 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d6b6321705947b6a395f77c11b1c6a5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:02.487994 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-59629ce5-996e-4bc1-89ae-6a750cea7171 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d6b6321705947b6a395f77c11b1c6a5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:02.495200 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1088/2161] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:08:01 2022] DELETE /identity/v3/users/5d6b6321705947b6a395f77c11b1c6a5 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:02.496385 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59629ce5-996e-4bc1-89ae-6a750cea7171 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:02.497887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-59629ce5-996e-4bc1-89ae-6a750cea7171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:02.577502 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-f1aba74c-8971-4a48-83ab-f659d25366d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8464dbd143a04815aa5019e244ee7e89'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:02.578105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-f1aba74c-8971-4a48-83ab-f659d25366d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8464dbd143a04815aa5019e244ee7e89'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:02.585139 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1074/2162] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:08:01 2022] DELETE /identity/v3/users/8464dbd143a04815aa5019e244ee7e89 => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:02.588723 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59629ce5-996e-4bc1-89ae-6a750cea7171 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:02.659175 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14e91e-d176-4a5a-9972-17bdda4f24dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:02.659838 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14e91e-d176-4a5a-9972-17bdda4f24dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:02.660237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14e91e-d176-4a5a-9972-17bdda4f24dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:02.660520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14e91e-d176-4a5a-9972-17bdda4f24dc admin admin] PATH_INFO: `/v3/users/f257210e7b8c45f2a4839678f0e429aa` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:02.661791 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14e91e-d176-4a5a-9972-17bdda4f24dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f257210e7b8c45f2a4839678f0e429aa)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:02.664002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14e91e-d176-4a5a-9972-17bdda4f24dc admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:02.863441 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aba74c-8971-4a48-83ab-f659d25366d2 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:02.863441 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f1aba74c-8971-4a48-83ab-f659d25366d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:02.897688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aba74c-8971-4a48-83ab-f659d25366d2 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:02.917278 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21756792-dda6-46c4-85d4-561f9bb1c28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:02.918410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21756792-dda6-46c4-85d4-561f9bb1c28c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:02.918656 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21756792-dda6-46c4-85d4-561f9bb1c28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:02.919336 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21756792-dda6-46c4-85d4-561f9bb1c28c admin admin] PATH_INFO: `/v3/projects/933ef2ab34a441b982b69839434d689e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:02.920773 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21756792-dda6-46c4-85d4-561f9bb1c28c admin admin] RBAC: Authorizing `identity:delete_project(project_id=933ef2ab34a441b982b69839434d689e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:02.922850 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21756792-dda6-46c4-85d4-561f9bb1c28c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:02.957511 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-de14e91e-d176-4a5a-9972-17bdda4f24dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f257210e7b8c45f2a4839678f0e429aa'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:02.980654 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-de14e91e-d176-4a5a-9972-17bdda4f24dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f257210e7b8c45f2a4839678f0e429aa'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:02.982703 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:02.992018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-de14e91e-d176-4a5a-9972-17bdda4f24dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f257210e7b8c45f2a4839678f0e429aa'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:03.098136 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-de14e91e-d176-4a5a-9972-17bdda4f24dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f257210e7b8c45f2a4839678f0e429aa'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:03.102148 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-de14e91e-d176-4a5a-9972-17bdda4f24dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f257210e7b8c45f2a4839678f0e429aa'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:03.120000 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1089/2163] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:08:02 2022] DELETE /identity/v3/users/f257210e7b8c45f2a4839678f0e429aa => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:03.137218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-21756792-dda6-46c4-85d4-561f9bb1c28c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '933ef2ab34a441b982b69839434d689e', 'request_id': 'req-21756792-dda6-46c4-85d4-561f9bb1c28c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:03.212630 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14e91e-d176-4a5a-9972-17bdda4f24dc admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:03.214428 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-de14e91e-d176-4a5a-9972-17bdda4f24dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:03.265417 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-21756792-dda6-46c4-85d4-561f9bb1c28c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '933ef2ab34a441b982b69839434d689e', 'request_id': 'req-21756792-dda6-46c4-85d4-561f9bb1c28c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:03.278979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-21756792-dda6-46c4-85d4-561f9bb1c28c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '933ef2ab34a441b982b69839434d689e', 'request_id': 'req-21756792-dda6-46c4-85d4-561f9bb1c28c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:03.311256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-21756792-dda6-46c4-85d4-561f9bb1c28c admin admin] The token cache is being invalidate because project 933ef2ab34a441b982b69839434d689e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:08:03.313771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-21756792-dda6-46c4-85d4-561f9bb1c28c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:03.313771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-21756792-dda6-46c4-85d4-561f9bb1c28c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:03.314238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1075/2164] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:08:02 2022] DELETE /identity/v3/projects/933ef2ab34a441b982b69839434d689e => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:08:03.319918 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50ca34-4bf2-4d3b-b204-e2c8473f8adc None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:03.320479 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50ca34-4bf2-4d3b-b204-e2c8473f8adc None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:03.321018 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50ca34-4bf2-4d3b-b204-e2c8473f8adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:03.344651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14e91e-d176-4a5a-9972-17bdda4f24dc admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:03.355996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ed50ca34-4bf2-4d3b-b204-e2c8473f8adc None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:03.357394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ed50ca34-4bf2-4d3b-b204-e2c8473f8adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:03.360853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee92b0-3f96-46df-be2a-acae0406ab98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:03.362818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee92b0-3f96-46df-be2a-acae0406ab98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:03.363295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee92b0-3f96-46df-be2a-acae0406ab98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:03.363754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee92b0-3f96-46df-be2a-acae0406ab98 admin admin] PATH_INFO: `/v3/projects/014fde5ff75449b9a9e6c8fe4630f4a9` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:03.365841 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee92b0-3f96-46df-be2a-acae0406ab98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=014fde5ff75449b9a9e6c8fe4630f4a9)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:03.374241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee92b0-3f96-46df-be2a-acae0406ab98 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:03.386230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1076/2165] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:03.390603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994a5b6-a6da-4409-b736-13b2e8ed021e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:03.391120 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994a5b6-a6da-4409-b736-13b2e8ed021e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:03.391966 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994a5b6-a6da-4409-b736-13b2e8ed021e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:03.422425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9994a5b6-a6da-4409-b736-13b2e8ed021e None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:03.424062 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9994a5b6-a6da-4409-b736-13b2e8ed021e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:03.437908 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1077/2166] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:08:03.441284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994a5b6-a6da-4409-b736-13b2e8ed021e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:03.459833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994a5b6-a6da-4409-b736-13b2e8ed021e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:03.467027 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584cf55-9a2a-4fa6-9276-57eed190292c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:03.468005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584cf55-9a2a-4fa6-9276-57eed190292c admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:03.468469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584cf55-9a2a-4fa6-9276-57eed190292c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:03.468822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584cf55-9a2a-4fa6-9276-57eed190292c admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:03.469692 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584cf55-9a2a-4fa6-9276-57eed190292c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:08:03.470193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584cf55-9a2a-4fa6-9276-57eed190292c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:03.472688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584cf55-9a2a-4fa6-9276-57eed190292c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:03.488848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1078/2167] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:08:03.493278 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584cf55-9a2a-4fa6-9276-57eed190292c admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:03.513633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584cf55-9a2a-4fa6-9276-57eed190292c admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:03.527523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f42da-d75f-4314-bfdd-5e8d2a1cc7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:03.528156 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f42da-d75f-4314-bfdd-5e8d2a1cc7a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:03.528445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f42da-d75f-4314-bfdd-5e8d2a1cc7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:03.528719 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f42da-d75f-4314-bfdd-5e8d2a1cc7a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:03.529384 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f42da-d75f-4314-bfdd-5e8d2a1cc7a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:03.531503 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f42da-d75f-4314-bfdd-5e8d2a1cc7a4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:03.576663 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-eaee92b0-3f96-46df-be2a-acae0406ab98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '014fde5ff75449b9a9e6c8fe4630f4a9', 'request_id': 'req-eaee92b0-3f96-46df-be2a-acae0406ab98'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:03.663649 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-eaee92b0-3f96-46df-be2a-acae0406ab98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '014fde5ff75449b9a9e6c8fe4630f4a9', 'request_id': 'req-eaee92b0-3f96-46df-be2a-acae0406ab98'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:03.671335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-eaee92b0-3f96-46df-be2a-acae0406ab98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '014fde5ff75449b9a9e6c8fe4630f4a9', 'request_id': 'req-eaee92b0-3f96-46df-be2a-acae0406ab98'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:03.678052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1079/2168] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:08:03 2022] POST /identity/v3/projects => generated 383 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:08:03.678844 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f42da-d75f-4314-bfdd-5e8d2a1cc7a4 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:03.684492 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-eaee92b0-3f96-46df-be2a-acae0406ab98 admin admin] The token cache is being invalidate because project 014fde5ff75449b9a9e6c8fe4630f4a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:08:03.685036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-eaee92b0-3f96-46df-be2a-acae0406ab98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:03.685754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-eaee92b0-3f96-46df-be2a-acae0406ab98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:03.688072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1090/2169] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:08:03 2022] DELETE /identity/v3/projects/014fde5ff75449b9a9e6c8fe4630f4a9 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:08:03.694239 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477112b8-9930-46f4-9600-d546e18dd723 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:03.694569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477112b8-9930-46f4-9600-d546e18dd723 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:03.695605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477112b8-9930-46f4-9600-d546e18dd723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:03.719333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-477112b8-9930-46f4-9600-d546e18dd723 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:03.720752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-477112b8-9930-46f4-9600-d546e18dd723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:03.739188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:03.760338 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f42da-d75f-4314-bfdd-5e8d2a1cc7a4 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:03.767724 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24098f9-9523-4a4f-abba-6aae3879d257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:03.768344 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24098f9-9523-4a4f-abba-6aae3879d257 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:03.768632 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24098f9-9523-4a4f-abba-6aae3879d257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:03.768909 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24098f9-9523-4a4f-abba-6aae3879d257 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:03.769652 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24098f9-9523-4a4f-abba-6aae3879d257 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:03.771975 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24098f9-9523-4a4f-abba-6aae3879d257 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:03.796281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1091/2170] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:08:03.802090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08b11b-b0f4-4393-b653-6b6548338067 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:03.802391 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08b11b-b0f4-4393-b653-6b6548338067 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:03.802674 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08b11b-b0f4-4393-b653-6b6548338067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:03.824508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-dd08b11b-b0f4-4393-b653-6b6548338067 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:03.825761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-dd08b11b-b0f4-4393-b653-6b6548338067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:03.836996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1092/2171] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:08:03.841851 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08b11b-b0f4-4393-b653-6b6548338067 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:03.855619 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08b11b-b0f4-4393-b653-6b6548338067 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:03.863606 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fa675-a403-4648-99d6-0e243e9f1872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:03.864666 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fa675-a403-4648-99d6-0e243e9f1872 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:03.865263 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fa675-a403-4648-99d6-0e243e9f1872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:03.865605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fa675-a403-4648-99d6-0e243e9f1872 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:03.866788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fa675-a403-4648-99d6-0e243e9f1872 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:08:03.867305 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fa675-a403-4648-99d6-0e243e9f1872 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:03.870835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fa675-a403-4648-99d6-0e243e9f1872 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:03.888887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1093/2172] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 12:08:03.892810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fa675-a403-4648-99d6-0e243e9f1872 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:03.907196 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fa675-a403-4648-99d6-0e243e9f1872 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:03.912896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5edea5-e931-44d4-844c-d15593e4485a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:03.913525 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5edea5-e931-44d4-844c-d15593e4485a admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:03.913804 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5edea5-e931-44d4-844c-d15593e4485a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:03.914111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5edea5-e931-44d4-844c-d15593e4485a admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:03.914752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5edea5-e931-44d4-844c-d15593e4485a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:03.916699 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5edea5-e931-44d4-844c-d15593e4485a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:04.392071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1080/2173] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:08:03 2022] POST /identity/v3/users => generated 292 bytes in 710 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:08:04.392071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24098f9-9523-4a4f-abba-6aae3879d257 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:04.392071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a24098f9-9523-4a4f-abba-6aae3879d257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:04.400813 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1094/2174] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:08:03 2022] POST /identity/v3/projects => generated 385 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:08:04.400813 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5edea5-e931-44d4-844c-d15593e4485a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:04.400813 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6a5edea5-e931-44d4-844c-d15593e4485a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:04.404546 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24098f9-9523-4a4f-abba-6aae3879d257 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:04.489667 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb44653-c9b4-4475-be44-f50b4ecfccfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:04.490779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb44653-c9b4-4475-be44-f50b4ecfccfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:04.491391 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb44653-c9b4-4475-be44-f50b4ecfccfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:04.491717 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb44653-c9b4-4475-be44-f50b4ecfccfe admin admin] PATH_INFO: `/v3/projects/2b1f002e95844f86a432acc1cda8840d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:04.493021 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb44653-c9b4-4475-be44-f50b4ecfccfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b1f002e95844f86a432acc1cda8840d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:04.508694 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb44653-c9b4-4475-be44-f50b4ecfccfe admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:04.509199 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5edea5-e931-44d4-844c-d15593e4485a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:04.509199 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d0770-caad-47d6-9f4d-43b6b401add6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:04.509199 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d0770-caad-47d6-9f4d-43b6b401add6 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:04.509199 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d0770-caad-47d6-9f4d-43b6b401add6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:04.509688 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d0770-caad-47d6-9f4d-43b6b401add6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:04.509688 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d0770-caad-47d6-9f4d-43b6b401add6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:04.509688 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d0770-caad-47d6-9f4d-43b6b401add6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:04.518806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1095/2175] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:04.518806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d0770-caad-47d6-9f4d-43b6b401add6 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:04.531335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d0770-caad-47d6-9f4d-43b6b401add6 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:04.535174 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2bd502-5031-4984-bd81-1a6360597880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:04.535925 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2bd502-5031-4984-bd81-1a6360597880 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:04.536267 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2bd502-5031-4984-bd81-1a6360597880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:04.536612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2bd502-5031-4984-bd81-1a6360597880 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:04.537308 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2bd502-5031-4984-bd81-1a6360597880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:04.539380 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2bd502-5031-4984-bd81-1a6360597880 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:04.551664 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:04.652581 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1096/2176] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:08:04 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:08:04.653351 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2bd502-5031-4984-bd81-1a6360597880 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:04.665303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2bd502-5031-4984-bd81-1a6360597880 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:04.670759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a02df0-9653-4106-9fed-27e265d3ed49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:04.671482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a02df0-9653-4106-9fed-27e265d3ed49 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:04.671768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a02df0-9653-4106-9fed-27e265d3ed49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:04.672056 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a02df0-9653-4106-9fed-27e265d3ed49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:04.672737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a02df0-9653-4106-9fed-27e265d3ed49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:04.674610 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a02df0-9653-4106-9fed-27e265d3ed49 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:04.685805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bcb44653-c9b4-4475-be44-f50b4ecfccfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b1f002e95844f86a432acc1cda8840d', 'request_id': 'req-bcb44653-c9b4-4475-be44-f50b4ecfccfe'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:04.691781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1097/2177] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:04.691781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a02df0-9653-4106-9fed-27e265d3ed49 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:04.698120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a02df0-9653-4106-9fed-27e265d3ed49 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:04.761117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c5fca-d3cf-4b76-a4b0-da6a8e4a7c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:04.761738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c5fca-d3cf-4b76-a4b0-da6a8e4a7c0c admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:04.762045 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c5fca-d3cf-4b76-a4b0-da6a8e4a7c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:04.762321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c5fca-d3cf-4b76-a4b0-da6a8e4a7c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:04.762968 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05c5fca-d3cf-4b76-a4b0-da6a8e4a7c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:04.765072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05c5fca-d3cf-4b76-a4b0-da6a8e4a7c0c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:04.775716 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1098/2178] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:04.776389 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c5fca-d3cf-4b76-a4b0-da6a8e4a7c0c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:04.776849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bcb44653-c9b4-4475-be44-f50b4ecfccfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b1f002e95844f86a432acc1cda8840d', 'request_id': 'req-bcb44653-c9b4-4475-be44-f50b4ecfccfe'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:04.787851 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c5fca-d3cf-4b76-a4b0-da6a8e4a7c0c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:04.789668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bcb44653-c9b4-4475-be44-f50b4ecfccfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b1f002e95844f86a432acc1cda8840d', 'request_id': 'req-bcb44653-c9b4-4475-be44-f50b4ecfccfe'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:04.793832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d27b39-f10b-4e6c-887a-1614b137dd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:04.794535 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d27b39-f10b-4e6c-887a-1614b137dd82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:04.794878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d27b39-f10b-4e6c-887a-1614b137dd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:04.795072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d27b39-f10b-4e6c-887a-1614b137dd82 admin admin] PATH_INFO: `/v3/projects/5b86ec91335c49aaa672aa85461d2936/users/31f3271adc36406fb303fb169020e740/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:04.814877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d27b39-f10b-4e6c-887a-1614b137dd82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b86ec91335c49aaa672aa85461d2936, user_id=31f3271adc36406fb303fb169020e740, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:04.817699 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d27b39-f10b-4e6c-887a-1614b137dd82 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:04.817840 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bcb44653-c9b4-4475-be44-f50b4ecfccfe admin admin] The token cache is being invalidate because project 2b1f002e95844f86a432acc1cda8840d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:08:04.821283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bcb44653-c9b4-4475-be44-f50b4ecfccfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:04.821283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bcb44653-c9b4-4475-be44-f50b4ecfccfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:04.821283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1081/2179] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:08:04 2022] DELETE /identity/v3/projects/2b1f002e95844f86a432acc1cda8840d => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:04.823043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb44653-c9b4-4475-be44-f50b4ecfccfe admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:04.825319 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-bcb44653-c9b4-4475-be44-f50b4ecfccfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:04.844462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb44653-c9b4-4475-be44-f50b4ecfccfe admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:04.850514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edbf41-c374-4578-b61b-95f2f7be5c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:04.850514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edbf41-c374-4578-b61b-95f2f7be5c50 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:04.850928 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edbf41-c374-4578-b61b-95f2f7be5c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:04.851413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edbf41-c374-4578-b61b-95f2f7be5c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:04.852026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26edbf41-c374-4578-b61b-95f2f7be5c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:04.853828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26edbf41-c374-4578-b61b-95f2f7be5c50 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:04.865180 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1082/2180] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:04.866669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2167509-083f-4b64-bee6-b40529a4e1ba None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:04.867019 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2167509-083f-4b64-bee6-b40529a4e1ba None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:04.867238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2167509-083f-4b64-bee6-b40529a4e1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:04.890879 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c2167509-083f-4b64-bee6-b40529a4e1ba None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:04.892121 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c2167509-083f-4b64-bee6-b40529a4e1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:04.900777 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1083/2181] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:04.901664 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2167509-083f-4b64-bee6-b40529a4e1ba None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:04.921815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2167509-083f-4b64-bee6-b40529a4e1ba None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:04.927340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017f554-7bdf-42ab-b622-aa5d6eb7b895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:04.928044 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017f554-7bdf-42ab-b622-aa5d6eb7b895 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:04.928348 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017f554-7bdf-42ab-b622-aa5d6eb7b895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:04.928615 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017f554-7bdf-42ab-b622-aa5d6eb7b895 admin admin] PATH_INFO: `/v3/projects/4dda01f4dfae4428b4ea737705d2edcd/users/a27bb57559b04fe686452b8c8d5ea041/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:04.947681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017f554-7bdf-42ab-b622-aa5d6eb7b895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dda01f4dfae4428b4ea737705d2edcd, user_id=a27bb57559b04fe686452b8c8d5ea041, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:04.951195 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017f554-7bdf-42ab-b622-aa5d6eb7b895 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:04.962724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1099/2182] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:08:04 2022] PUT /identity/v3/projects/5b86ec91335c49aaa672aa85461d2936/users/31f3271adc36406fb303fb169020e740/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:04.965265 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2a08b-7fb1-45b2-9535-b2d9e36363cd None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:04.965265 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2a08b-7fb1-45b2-9535-b2d9e36363cd None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:04.965265 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2a08b-7fb1-45b2-9535-b2d9e36363cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:04.986907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-27e2a08b-7fb1-45b2-9535-b2d9e36363cd None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:04.999419 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-27e2a08b-7fb1-45b2-9535-b2d9e36363cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:05.036079 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1084/2183] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:08:04 2022] PUT /identity/v3/projects/4dda01f4dfae4428b4ea737705d2edcd/users/a27bb57559b04fe686452b8c8d5ea041/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:05.039769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07765f66-f726-4696-8596-1d8e97bc2b28 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:05.039769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07765f66-f726-4696-8596-1d8e97bc2b28 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:05.039769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07765f66-f726-4696-8596-1d8e97bc2b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:05.058859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1100/2184] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:05.060919 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab803314-0f10-4976-bf1f-922073424990 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:05.061955 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab803314-0f10-4976-bf1f-922073424990 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:05.062578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab803314-0f10-4976-bf1f-922073424990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:05.064496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-07765f66-f726-4696-8596-1d8e97bc2b28 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:05.066345 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-07765f66-f726-4696-8596-1d8e97bc2b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:05.101850 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ab803314-0f10-4976-bf1f-922073424990 None None] MFA Rules not processed for user `a27bb57559b04fe686452b8c8d5ea041`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:05.103315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ab803314-0f10-4976-bf1f-922073424990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:05.143625 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1085/2185] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:05.144508 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07765f66-f726-4696-8596-1d8e97bc2b28 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:05.162852 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07765f66-f726-4696-8596-1d8e97bc2b28 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:05.168208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dde208-665e-4195-9cf9-6b2965850ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:05.168829 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dde208-665e-4195-9cf9-6b2965850ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:05.169115 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dde208-665e-4195-9cf9-6b2965850ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:05.169383 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dde208-665e-4195-9cf9-6b2965850ff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:05.170078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dde208-665e-4195-9cf9-6b2965850ff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:08:05.170446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dde208-665e-4195-9cf9-6b2965850ff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:05.172150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dde208-665e-4195-9cf9-6b2965850ff7 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:05.188846 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1086/2186] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:08:05.189830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dde208-665e-4195-9cf9-6b2965850ff7 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:05.192433 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f0dde208-665e-4195-9cf9-6b2965850ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:05.196970 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1101/2187] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:05.206051 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab803314-0f10-4976-bf1f-922073424990 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:05.214340 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab803314-0f10-4976-bf1f-922073424990 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:05.219830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59193b-2c7a-46ad-9a4f-679a7997d0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:05.220581 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59193b-2c7a-46ad-9a4f-679a7997d0ae admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:05.220941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59193b-2c7a-46ad-9a4f-679a7997d0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:05.221287 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59193b-2c7a-46ad-9a4f-679a7997d0ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:05.222191 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59193b-2c7a-46ad-9a4f-679a7997d0ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:05.224256 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59193b-2c7a-46ad-9a4f-679a7997d0ae admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:05.279417 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dde208-665e-4195-9cf9-6b2965850ff7 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:05.283521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc1fe4-1b82-4034-9c96-526d7eaf924b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:05.284807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc1fe4-1b82-4034-9c96-526d7eaf924b service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:05.285088 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc1fe4-1b82-4034-9c96-526d7eaf924b service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:05.285358 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc1fe4-1b82-4034-9c96-526d7eaf924b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:05.297280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc1fe4-1b82-4034-9c96-526d7eaf924b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:05.298814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc1fe4-1b82-4034-9c96-526d7eaf924b service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:05.309322 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1087/2188] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:08:05.310221 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc1fe4-1b82-4034-9c96-526d7eaf924b service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:05.312242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b4cc1fe4-1b82-4034-9c96-526d7eaf924b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:05.354145 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1102/2189] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:08:05 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:08:05.358358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59193b-2c7a-46ad-9a4f-679a7997d0ae admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:05.392624 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc1fe4-1b82-4034-9c96-526d7eaf924b service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:05.440183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:05.470782 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59193b-2c7a-46ad-9a4f-679a7997d0ae admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:05.478819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf808e-3650-4aa2-a4d6-0e8b776224ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:05.479820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf808e-3650-4aa2-a4d6-0e8b776224ba admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:05.480273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf808e-3650-4aa2-a4d6-0e8b776224ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:05.480800 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf808e-3650-4aa2-a4d6-0e8b776224ba admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:05.482470 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf808e-3650-4aa2-a4d6-0e8b776224ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:05.485420 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15650162-4302-469e-aa20-07364827d422 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:05.487024 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf808e-3650-4aa2-a4d6-0e8b776224ba admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:05.487610 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15650162-4302-469e-aa20-07364827d422 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:05.488204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15650162-4302-469e-aa20-07364827d422 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:05.488740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15650162-4302-469e-aa20-07364827d422 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:05.561975 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15650162-4302-469e-aa20-07364827d422 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:05.563490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15650162-4302-469e-aa20-07364827d422 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:05.573630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1088/2190] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:08:05.599320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1103/2191] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:08:05 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:08:05.600540 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15650162-4302-469e-aa20-07364827d422 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:05.613158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15650162-4302-469e-aa20-07364827d422 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:05.620352 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:05.629249 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c9b7c-46c6-4052-b1f3-bf2f3cb2db63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:05.630537 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c9b7c-46c6-4052-b1f3-bf2f3cb2db63 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:05.631530 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c9b7c-46c6-4052-b1f3-bf2f3cb2db63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:05.631999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c9b7c-46c6-4052-b1f3-bf2f3cb2db63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:05.633225 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c9b7c-46c6-4052-b1f3-bf2f3cb2db63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:05.635792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c9b7c-46c6-4052-b1f3-bf2f3cb2db63 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:05.663726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1089/2192] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:05.668132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf808e-3650-4aa2-a4d6-0e8b776224ba admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:05.681048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf808e-3650-4aa2-a4d6-0e8b776224ba admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:05.687158 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d40005-198a-46df-8dc3-5ba107b11913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:05.687819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d40005-198a-46df-8dc3-5ba107b11913 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:05.688099 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d40005-198a-46df-8dc3-5ba107b11913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:05.688804 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d40005-198a-46df-8dc3-5ba107b11913 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:05.689504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d40005-198a-46df-8dc3-5ba107b11913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:05.691538 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d40005-198a-46df-8dc3-5ba107b11913 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:05.708098 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1104/2193] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:05.714144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c9b7c-46c6-4052-b1f3-bf2f3cb2db63 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:05.731761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c9b7c-46c6-4052-b1f3-bf2f3cb2db63 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:05.744490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea796b39-268d-4ab2-b696-4c70f0ab39a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:05.745229 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea796b39-268d-4ab2-b696-4c70f0ab39a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:05.745530 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea796b39-268d-4ab2-b696-4c70f0ab39a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:05.745801 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea796b39-268d-4ab2-b696-4c70f0ab39a8 admin admin] PATH_INFO: `/v3/projects/ca512c2e96e64742a6426b2990956b99/users/a13106affcb943a8a42b2f1edb3143bb/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:05.763629 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea796b39-268d-4ab2-b696-4c70f0ab39a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca512c2e96e64742a6426b2990956b99, user_id=a13106affcb943a8a42b2f1edb3143bb, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:05.766608 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea796b39-268d-4ab2-b696-4c70f0ab39a8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:05.859924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1090/2194] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:08:05 2022] PUT /identity/v3/projects/ca512c2e96e64742a6426b2990956b99/users/a13106affcb943a8a42b2f1edb3143bb/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:08:05.876138 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779dcef-dd4e-466b-984b-2a5081f07f7e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:05.876434 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779dcef-dd4e-466b-984b-2a5081f07f7e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:05.877030 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779dcef-dd4e-466b-984b-2a5081f07f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:05.919621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1779dcef-dd4e-466b-984b-2a5081f07f7e None None] MFA Rules not processed for user `a13106affcb943a8a42b2f1edb3143bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:05.920443 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1779dcef-dd4e-466b-984b-2a5081f07f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:06.042142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1105/2195] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:06.047434 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea796b39-268d-4ab2-b696-4c70f0ab39a8 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:06.133741 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:06.156021 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea796b39-268d-4ab2-b696-4c70f0ab39a8 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:06.164163 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1cdf10-6c24-46e2-8521-a0dddb099ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:06.164833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cdf10-6c24-46e2-8521-a0dddb099ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:06.164960 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cdf10-6c24-46e2-8521-a0dddb099ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:06.165257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cdf10-6c24-46e2-8521-a0dddb099ce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:06.166073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1cdf10-6c24-46e2-8521-a0dddb099ce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:06.168127 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1cdf10-6c24-46e2-8521-a0dddb099ce9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:06.291935 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1091/2196] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:08:06 2022] POST /identity/v3/projects => generated 385 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:08:06.296177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779dcef-dd4e-466b-984b-2a5081f07f7e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:06.311203 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779dcef-dd4e-466b-984b-2a5081f07f7e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:06.379278 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd92b88-cf0a-4c36-8cdb-5747234b158a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:06.380167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd92b88-cf0a-4c36-8cdb-5747234b158a admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:06.380563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd92b88-cf0a-4c36-8cdb-5747234b158a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:06.380966 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd92b88-cf0a-4c36-8cdb-5747234b158a admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:06.381660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd92b88-cf0a-4c36-8cdb-5747234b158a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:06.383734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd92b88-cf0a-4c36-8cdb-5747234b158a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:06.430309 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:06.547135 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1106/2197] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:08:06 2022] POST /identity/v3/users => generated 293 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:08:06.552060 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1cdf10-6c24-46e2-8521-a0dddb099ce9 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:06.563530 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1cdf10-6c24-46e2-8521-a0dddb099ce9 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:06.569884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e9bb6-11a1-4309-9418-4e936adc0c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:06.570593 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e9bb6-11a1-4309-9418-4e936adc0c1a admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:06.570877 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e9bb6-11a1-4309-9418-4e936adc0c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:06.571152 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e9bb6-11a1-4309-9418-4e936adc0c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:06.571827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e9bb6-11a1-4309-9418-4e936adc0c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:06.573743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e9bb6-11a1-4309-9418-4e936adc0c1a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:06.586903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1092/2198] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:06.592284 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd92b88-cf0a-4c36-8cdb-5747234b158a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:06.627079 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd92b88-cf0a-4c36-8cdb-5747234b158a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:06.640713 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fda0eb-dada-42aa-b081-8ce35deafb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:06.641901 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fda0eb-dada-42aa-b081-8ce35deafb1a admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:06.642361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fda0eb-dada-42aa-b081-8ce35deafb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:06.642796 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fda0eb-dada-42aa-b081-8ce35deafb1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:06.643756 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fda0eb-dada-42aa-b081-8ce35deafb1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:06.647562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fda0eb-dada-42aa-b081-8ce35deafb1a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:06.666960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1107/2199] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:06.678441 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e9bb6-11a1-4309-9418-4e936adc0c1a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:06.694552 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e9bb6-11a1-4309-9418-4e936adc0c1a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:06.701115 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d37862-f0aa-452d-bd4b-e7da3592e03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:06.702612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d37862-f0aa-452d-bd4b-e7da3592e03d admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:06.703084 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d37862-f0aa-452d-bd4b-e7da3592e03d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:06.703452 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d37862-f0aa-452d-bd4b-e7da3592e03d admin admin] PATH_INFO: `/v3/projects/d070ecb201ca4812a9a7441ae564596f/users/1c5f791b1e2b4633b3ccb66cd304850f/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:06.728414 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d37862-f0aa-452d-bd4b-e7da3592e03d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d070ecb201ca4812a9a7441ae564596f, user_id=1c5f791b1e2b4633b3ccb66cd304850f, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:06.733898 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d37862-f0aa-452d-bd4b-e7da3592e03d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:06.844875 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1093/2200] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:08:06 2022] PUT /identity/v3/projects/d070ecb201ca4812a9a7441ae564596f/users/1c5f791b1e2b4633b3ccb66cd304850f/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:06.918537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce441e-763b-42e7-a77b-42ab79ee9c66 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:06.918855 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce441e-763b-42e7-a77b-42ab79ee9c66 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:06.919153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce441e-763b-42e7-a77b-42ab79ee9c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:06.958526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d0ce441e-763b-42e7-a77b-42ab79ee9c66 None None] MFA Rules not processed for user `1c5f791b1e2b4633b3ccb66cd304850f`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:06.960008 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d0ce441e-763b-42e7-a77b-42ab79ee9c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:07.062895 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1108/2201] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:08:07.067395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d37862-f0aa-452d-bd4b-e7da3592e03d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:07.142701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:07.173231 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d37862-f0aa-452d-bd4b-e7da3592e03d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:07.194622 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdb377-d518-4add-9932-f120189d58da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:07.195932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdb377-d518-4add-9932-f120189d58da admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:07.196339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdb377-d518-4add-9932-f120189d58da admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:07.196688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdb377-d518-4add-9932-f120189d58da admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:07.197645 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdb377-d518-4add-9932-f120189d58da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:07.200919 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdb377-d518-4add-9932-f120189d58da admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:07.308338 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1094/2202] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:08:07 2022] POST /identity/v3/projects => generated 385 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:08:07.312261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce441e-763b-42e7-a77b-42ab79ee9c66 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:07.346799 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce441e-763b-42e7-a77b-42ab79ee9c66 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:07.426578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:07.456070 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca874b-64dc-4589-8b29-94a369ab62d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:07.457059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca874b-64dc-4589-8b29-94a369ab62d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:07.461347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca874b-64dc-4589-8b29-94a369ab62d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:07.461347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca874b-64dc-4589-8b29-94a369ab62d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:07.461347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca874b-64dc-4589-8b29-94a369ab62d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:07.461347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca874b-64dc-4589-8b29-94a369ab62d1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:07.573275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1109/2203] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:08:07 2022] POST /identity/v3/users => generated 299 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:08:07.574626 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdb377-d518-4add-9932-f120189d58da admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:07.588427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdb377-d518-4add-9932-f120189d58da admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:07.599421 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a365ef5-a78c-4c13-b2e9-4f133e321d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:07.603571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a365ef5-a78c-4c13-b2e9-4f133e321d82 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:07.603571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a365ef5-a78c-4c13-b2e9-4f133e321d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:07.603571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a365ef5-a78c-4c13-b2e9-4f133e321d82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:07.603571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a365ef5-a78c-4c13-b2e9-4f133e321d82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:07.604204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a365ef5-a78c-4c13-b2e9-4f133e321d82 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:07.620933 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1095/2204] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:07.622379 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca874b-64dc-4589-8b29-94a369ab62d1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:07.641613 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca874b-64dc-4589-8b29-94a369ab62d1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:07.642514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288cd2b8-4460-4598-83b1-75a30c20d116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:07.643415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cd2b8-4460-4598-83b1-75a30c20d116 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:07.643849 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cd2b8-4460-4598-83b1-75a30c20d116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:07.644247 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cd2b8-4460-4598-83b1-75a30c20d116 admin admin] PATH_INFO: `/v3/projects/30f508d051884ca299571768f1f6d8b6/users/0afebe6ff72440b48d7d999ace053e3a/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:07.671650 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288cd2b8-4460-4598-83b1-75a30c20d116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f508d051884ca299571768f1f6d8b6, user_id=0afebe6ff72440b48d7d999ace053e3a, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:07.674778 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288cd2b8-4460-4598-83b1-75a30c20d116 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:07.800176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1110/2205] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:08:07 2022] PUT /identity/v3/projects/30f508d051884ca299571768f1f6d8b6/users/0afebe6ff72440b48d7d999ace053e3a/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:07.802188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a365ef5-a78c-4c13-b2e9-4f133e321d82 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:07.817086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a365ef5-a78c-4c13-b2e9-4f133e321d82 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:07.976653 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc535f0-7074-4b09-8527-9434bf75165d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:07.977734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc535f0-7074-4b09-8527-9434bf75165d admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:07.978367 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc535f0-7074-4b09-8527-9434bf75165d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:07.978794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc535f0-7074-4b09-8527-9434bf75165d admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:07.980399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc535f0-7074-4b09-8527-9434bf75165d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:07.983639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc535f0-7074-4b09-8527-9434bf75165d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:08.006301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1096/2206] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:08.007848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288cd2b8-4460-4598-83b1-75a30c20d116 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:08.043249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288cd2b8-4460-4598-83b1-75a30c20d116 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:08.052633 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b751d3-cd93-4d63-a517-115772649631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:08.053982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b751d3-cd93-4d63-a517-115772649631 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:08.058137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b751d3-cd93-4d63-a517-115772649631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:08.058137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b751d3-cd93-4d63-a517-115772649631 admin admin] PATH_INFO: `/v3/projects/30f508d051884ca299571768f1f6d8b6/users/0afebe6ff72440b48d7d999ace053e3a/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:08.059081 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b751d3-cd93-4d63-a517-115772649631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f508d051884ca299571768f1f6d8b6, user_id=0afebe6ff72440b48d7d999ace053e3a, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:08.063486 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b751d3-cd93-4d63-a517-115772649631 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:08.087368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1111/2207] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:08:08 2022] PUT /identity/v3/projects/30f508d051884ca299571768f1f6d8b6/users/0afebe6ff72440b48d7d999ace053e3a/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:08.099614 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95400c53-6acb-439d-b33c-3bf2777f4cd4 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:08.100655 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95400c53-6acb-439d-b33c-3bf2777f4cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:08.100655 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95400c53-6acb-439d-b33c-3bf2777f4cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:08.140120 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-95400c53-6acb-439d-b33c-3bf2777f4cd4 None None] MFA Rules not processed for user `0afebe6ff72440b48d7d999ace053e3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:08.141042 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-95400c53-6acb-439d-b33c-3bf2777f4cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:08.259712 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b751d3-cd93-4d63-a517-115772649631 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:08.265916 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1097/2208] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:08.267047 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-87b751d3-cd93-4d63-a517-115772649631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:08.283595 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95400c53-6acb-439d-b33c-3bf2777f4cd4 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:08.285890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-95400c53-6acb-439d-b33c-3bf2777f4cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:08.379175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:08.402818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95400c53-6acb-439d-b33c-3bf2777f4cd4 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:08.407304 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d07db0-e294-415c-93e2-105a044088c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:08.408105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d07db0-e294-415c-93e2-105a044088c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:08.409008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d07db0-e294-415c-93e2-105a044088c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:08.409008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d07db0-e294-415c-93e2-105a044088c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:08.423305 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b751d3-cd93-4d63-a517-115772649631 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:08.430106 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd2141-0dac-458f-842c-1598040a5270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:08.430784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd2141-0dac-458f-842c-1598040a5270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:08.431093 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd2141-0dac-458f-842c-1598040a5270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:08.431602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd2141-0dac-458f-842c-1598040a5270 admin admin] PATH_INFO: `/v3/users/820db9f8b3f2443a932f844758bba703` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:08.434639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd2141-0dac-458f-842c-1598040a5270 admin admin] RBAC: Authorizing `identity:delete_user(user_id=820db9f8b3f2443a932f844758bba703)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:08.437172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd2141-0dac-458f-842c-1598040a5270 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:08.527100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d07db0-e294-415c-93e2-105a044088c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:08.529814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d07db0-e294-415c-93e2-105a044088c5 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:08.560256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1098/2209] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:08:08.767916 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fadd2141-0dac-458f-842c-1598040a5270 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '820db9f8b3f2443a932f844758bba703'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:08.770941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:08.794801 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fadd2141-0dac-458f-842c-1598040a5270 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '820db9f8b3f2443a932f844758bba703'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:08.808424 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fadd2141-0dac-458f-842c-1598040a5270 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '820db9f8b3f2443a932f844758bba703'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:08.909668 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fadd2141-0dac-458f-842c-1598040a5270 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '820db9f8b3f2443a932f844758bba703'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:08.910908 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fadd2141-0dac-458f-842c-1598040a5270 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '820db9f8b3f2443a932f844758bba703'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:08.923326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1112/2210] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:08:08 2022] DELETE /identity/v3/users/820db9f8b3f2443a932f844758bba703 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:09.274715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d07db0-e294-415c-93e2-105a044088c5 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:09.287323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d07db0-e294-415c-93e2-105a044088c5 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:09.369022 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5df4d-de70-4b17-adb4-43a72ab5507a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:09.369022 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5df4d-de70-4b17-adb4-43a72ab5507a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:09.369022 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5df4d-de70-4b17-adb4-43a72ab5507a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:09.369022 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5df4d-de70-4b17-adb4-43a72ab5507a admin admin] PATH_INFO: `/v3/projects/6c9979087f9746b59bd1c2b4e0dbcd4b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:09.371084 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5df4d-de70-4b17-adb4-43a72ab5507a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c9979087f9746b59bd1c2b4e0dbcd4b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:09.374946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5df4d-de70-4b17-adb4-43a72ab5507a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:09.573971 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3cd5df4d-de70-4b17-adb4-43a72ab5507a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c9979087f9746b59bd1c2b4e0dbcd4b', 'request_id': 'req-3cd5df4d-de70-4b17-adb4-43a72ab5507a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:09.681137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3cd5df4d-de70-4b17-adb4-43a72ab5507a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c9979087f9746b59bd1c2b4e0dbcd4b', 'request_id': 'req-3cd5df4d-de70-4b17-adb4-43a72ab5507a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:09.690230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3cd5df4d-de70-4b17-adb4-43a72ab5507a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c9979087f9746b59bd1c2b4e0dbcd4b', 'request_id': 'req-3cd5df4d-de70-4b17-adb4-43a72ab5507a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:09.730407 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3cd5df4d-de70-4b17-adb4-43a72ab5507a admin admin] The token cache is being invalidate because project 6c9979087f9746b59bd1c2b4e0dbcd4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:08:09.731161 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3cd5df4d-de70-4b17-adb4-43a72ab5507a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:09.733946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3cd5df4d-de70-4b17-adb4-43a72ab5507a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:09.737459 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1099/2211] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:08:09 2022] DELETE /identity/v3/projects/6c9979087f9746b59bd1c2b4e0dbcd4b => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:09.742005 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd2141-0dac-458f-842c-1598040a5270 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:09.746318 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fadd2141-0dac-458f-842c-1598040a5270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:09.767192 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd2141-0dac-458f-842c-1598040a5270 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:09.884090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d90dcd-1f8d-469e-8b52-9931730a2747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:09.885287 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d90dcd-1f8d-469e-8b52-9931730a2747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:09.885633 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d90dcd-1f8d-469e-8b52-9931730a2747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:09.886049 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d90dcd-1f8d-469e-8b52-9931730a2747 admin admin] PATH_INFO: `/v3/users/c2c62b6ec2b74a548ead44ae26afa2a5` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:09.887376 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d90dcd-1f8d-469e-8b52-9931730a2747 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2c62b6ec2b74a548ead44ae26afa2a5)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:09.889943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d90dcd-1f8d-469e-8b52-9931730a2747 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:10.332891 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-01d90dcd-1f8d-469e-8b52-9931730a2747 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2c62b6ec2b74a548ead44ae26afa2a5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:10.352192 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-01d90dcd-1f8d-469e-8b52-9931730a2747 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2c62b6ec2b74a548ead44ae26afa2a5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:10.363228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-01d90dcd-1f8d-469e-8b52-9931730a2747 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2c62b6ec2b74a548ead44ae26afa2a5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:10.451021 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-01d90dcd-1f8d-469e-8b52-9931730a2747 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2c62b6ec2b74a548ead44ae26afa2a5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:10.452749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-01d90dcd-1f8d-469e-8b52-9931730a2747 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2c62b6ec2b74a548ead44ae26afa2a5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:10.462837 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1113/2212] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:08:09 2022] DELETE /identity/v3/users/c2c62b6ec2b74a548ead44ae26afa2a5 => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:10.871223 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5df4d-de70-4b17-adb4-43a72ab5507a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:10.872769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3cd5df4d-de70-4b17-adb4-43a72ab5507a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:10.887139 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5df4d-de70-4b17-adb4-43a72ab5507a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:10.909053 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:10.960989 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1569030-07e3-4a85-a080-27586133ef02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:10.961328 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1569030-07e3-4a85-a080-27586133ef02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:10.961797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1569030-07e3-4a85-a080-27586133ef02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:10.962107 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1569030-07e3-4a85-a080-27586133ef02 admin admin] PATH_INFO: `/v3/projects/205377aab8864eed98123c393006b6e0` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:10.963374 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1569030-07e3-4a85-a080-27586133ef02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=205377aab8864eed98123c393006b6e0)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:10.965372 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1569030-07e3-4a85-a080-27586133ef02 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:11.105233 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a1569030-07e3-4a85-a080-27586133ef02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '205377aab8864eed98123c393006b6e0', 'request_id': 'req-a1569030-07e3-4a85-a080-27586133ef02'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:11.189412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a1569030-07e3-4a85-a080-27586133ef02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '205377aab8864eed98123c393006b6e0', 'request_id': 'req-a1569030-07e3-4a85-a080-27586133ef02'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:11.196456 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a1569030-07e3-4a85-a080-27586133ef02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '205377aab8864eed98123c393006b6e0', 'request_id': 'req-a1569030-07e3-4a85-a080-27586133ef02'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:11.209458 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a1569030-07e3-4a85-a080-27586133ef02 admin admin] The token cache is being invalidate because project 205377aab8864eed98123c393006b6e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:08:11.209833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a1569030-07e3-4a85-a080-27586133ef02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:11.210587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a1569030-07e3-4a85-a080-27586133ef02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:11.212483 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1100/2213] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:08:10 2022] DELETE /identity/v3/projects/205377aab8864eed98123c393006b6e0 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:11.217336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf72963-54b1-4324-ac77-966b4262e581 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:11.218353 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf72963-54b1-4324-ac77-966b4262e581 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:11.218574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf72963-54b1-4324-ac77-966b4262e581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:11.240432 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-faf72963-54b1-4324-ac77-966b4262e581 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:11.241780 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-faf72963-54b1-4324-ac77-966b4262e581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:11.296415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:11.322197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1114/2214] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:11.330845 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1c05a-e6ce-40ec-a7ca-7c0dd2fc490c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:11.331166 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1c05a-e6ce-40ec-a7ca-7c0dd2fc490c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:11.331503 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1c05a-e6ce-40ec-a7ca-7c0dd2fc490c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:11.365130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c6e1c05a-e6ce-40ec-a7ca-7c0dd2fc490c None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:11.366714 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c6e1c05a-e6ce-40ec-a7ca-7c0dd2fc490c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:11.376523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1101/2215] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:11.381656 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf72963-54b1-4324-ac77-966b4262e581 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:11.392687 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf72963-54b1-4324-ac77-966b4262e581 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:11.398126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106953d-18e6-46ab-b108-f6e4f36ebbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:11.399264 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106953d-18e6-46ab-b108-f6e4f36ebbc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:11.399576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106953d-18e6-46ab-b108-f6e4f36ebbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:11.399903 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106953d-18e6-46ab-b108-f6e4f36ebbc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:11.400559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8106953d-18e6-46ab-b108-f6e4f36ebbc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:08:11.400871 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8106953d-18e6-46ab-b108-f6e4f36ebbc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:11.402519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8106953d-18e6-46ab-b108-f6e4f36ebbc7 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:11.420393 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1115/2216] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:08:11.421184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1c05a-e6ce-40ec-a7ca-7c0dd2fc490c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:11.439141 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1c05a-e6ce-40ec-a7ca-7c0dd2fc490c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:11.444855 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43879bd9-7e85-4d06-975c-55f4a17ee4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:11.445788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43879bd9-7e85-4d06-975c-55f4a17ee4b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:11.446263 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43879bd9-7e85-4d06-975c-55f4a17ee4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:11.446690 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43879bd9-7e85-4d06-975c-55f4a17ee4b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:11.447540 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43879bd9-7e85-4d06-975c-55f4a17ee4b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:11.450252 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43879bd9-7e85-4d06-975c-55f4a17ee4b9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:11.557458 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1102/2217] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:08:11 2022] POST /identity/v3/projects => generated 367 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:08:11.566298 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106953d-18e6-46ab-b108-f6e4f36ebbc7 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:11.573230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106953d-18e6-46ab-b108-f6e4f36ebbc7 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:11.677490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d883183-a8b9-42bb-970f-c16ecf728a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:11.678172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d883183-a8b9-42bb-970f-c16ecf728a7c admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:11.678460 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d883183-a8b9-42bb-970f-c16ecf728a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:11.678933 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d883183-a8b9-42bb-970f-c16ecf728a7c admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:11.679973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d883183-a8b9-42bb-970f-c16ecf728a7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:11.682254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d883183-a8b9-42bb-970f-c16ecf728a7c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:11.773490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1116/2218] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:08:11 2022] POST /identity/v3/users => generated 284 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:08:11.778376 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43879bd9-7e85-4d06-975c-55f4a17ee4b9 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:11.791767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43879bd9-7e85-4d06-975c-55f4a17ee4b9 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:11.800046 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0ce0f-a9c1-48da-8d99-ea1bfc519a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:11.801141 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0ce0f-a9c1-48da-8d99-ea1bfc519a2e admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:11.802200 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0ce0f-a9c1-48da-8d99-ea1bfc519a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:11.802683 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0ce0f-a9c1-48da-8d99-ea1bfc519a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:11.803571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0ce0f-a9c1-48da-8d99-ea1bfc519a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:11.806798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0ce0f-a9c1-48da-8d99-ea1bfc519a2e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:11.824340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1103/2219] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:11.830986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d883183-a8b9-42bb-970f-c16ecf728a7c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:11.844991 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d883183-a8b9-42bb-970f-c16ecf728a7c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:11.852507 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151987b-94f7-42bd-abd4-43ccd8b932b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:11.853437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151987b-94f7-42bd-abd4-43ccd8b932b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:11.854319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151987b-94f7-42bd-abd4-43ccd8b932b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:11.854790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151987b-94f7-42bd-abd4-43ccd8b932b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:11.856205 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151987b-94f7-42bd-abd4-43ccd8b932b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:11.858462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151987b-94f7-42bd-abd4-43ccd8b932b4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:11.875992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1117/2220] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:11.884916 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0ce0f-a9c1-48da-8d99-ea1bfc519a2e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:11.902479 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0ce0f-a9c1-48da-8d99-ea1bfc519a2e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:11.917817 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ef803-72f9-48fd-9a8c-779696e5037a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:11.918517 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ef803-72f9-48fd-9a8c-779696e5037a admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:11.918799 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ef803-72f9-48fd-9a8c-779696e5037a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:11.919077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ef803-72f9-48fd-9a8c-779696e5037a admin admin] PATH_INFO: `/v3/projects/f6192727aecd471da46a4e01b07c3a39/users/f41074ef6a544113b744dd90ed4e77bf/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:11.945978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ef803-72f9-48fd-9a8c-779696e5037a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6192727aecd471da46a4e01b07c3a39, user_id=f41074ef6a544113b744dd90ed4e77bf, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:11.948825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ef803-72f9-48fd-9a8c-779696e5037a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:12.056728 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1104/2221] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:08:11 2022] PUT /identity/v3/projects/f6192727aecd471da46a4e01b07c3a39/users/f41074ef6a544113b744dd90ed4e77bf/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:12.058220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0569d-9e47-415b-bd4f-703adef1ee1a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:12.058757 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0569d-9e47-415b-bd4f-703adef1ee1a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:12.059235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0569d-9e47-415b-bd4f-703adef1ee1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:12.102507 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-acc0569d-9e47-415b-bd4f-703adef1ee1a None None] MFA Rules not processed for user `f41074ef6a544113b744dd90ed4e77bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:12.111398 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-acc0569d-9e47-415b-bd4f-703adef1ee1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:12.219855 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1118/2222] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:08:12.228519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ef803-72f9-48fd-9a8c-779696e5037a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:12.230300 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-394ef803-72f9-48fd-9a8c-779696e5037a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:12.310767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ef803-72f9-48fd-9a8c-779696e5037a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:12.314516 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62082c80-2f11-4cfe-b6e3-b53f751b59ad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b05d0e65e9645f3810a6d96c8fed9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:12.315151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62082c80-2f11-4cfe-b6e3-b53f751b59ad service glance] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:12.315435 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62082c80-2f11-4cfe-b6e3-b53f751b59ad service glance] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:12.315716 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62082c80-2f11-4cfe-b6e3-b53f751b59ad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:12.328526 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62082c80-2f11-4cfe-b6e3-b53f751b59ad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:12.330032 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62082c80-2f11-4cfe-b6e3-b53f751b59ad service glance] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:12.341213 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1105/2223] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:08:12.392904 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:12.454942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d2f57-d4b7-40f1-8475-bb327d0d928f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:12.455235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d2f57-d4b7-40f1-8475-bb327d0d928f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:12.455515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d2f57-d4b7-40f1-8475-bb327d0d928f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:12.499258 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-267d2f57-d4b7-40f1-8475-bb327d0d928f None None] MFA Rules not processed for user `31f3271adc36406fb303fb169020e740`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:12.500541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-267d2f57-d4b7-40f1-8475-bb327d0d928f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:12.542765 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:12.591718 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1119/2224] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:12.613645 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62082c80-2f11-4cfe-b6e3-b53f751b59ad service glance] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:12.615576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-62082c80-2f11-4cfe-b6e3-b53f751b59ad service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:12.718477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62082c80-2f11-4cfe-b6e3-b53f751b59ad service glance] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:12.724114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285aa3f-38ce-439b-adfe-a2542d7b7c40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:12.724744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285aa3f-38ce-439b-adfe-a2542d7b7c40 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:12.725022 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285aa3f-38ce-439b-adfe-a2542d7b7c40 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:12.725295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285aa3f-38ce-439b-adfe-a2542d7b7c40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:12.734522 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a285aa3f-38ce-439b-adfe-a2542d7b7c40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:12.736192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a285aa3f-38ce-439b-adfe-a2542d7b7c40 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:12.748498 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1106/2225] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:08:16.362658 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b636a9-9e07-4a97-b61d-4dde10b75b7e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:16.363177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b636a9-9e07-4a97-b61d-4dde10b75b7e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:16.363390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b636a9-9e07-4a97-b61d-4dde10b75b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:16.383785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-76b636a9-9e07-4a97-b61d-4dde10b75b7e None None] MFA Rules not processed for user `31f3271adc36406fb303fb169020e740`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:16.385081 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-76b636a9-9e07-4a97-b61d-4dde10b75b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:16.403500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1120/2226] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:17.694298 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285aa3f-38ce-439b-adfe-a2542d7b7c40 service nova] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:17.695788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a285aa3f-38ce-439b-adfe-a2542d7b7c40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:17.763449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285aa3f-38ce-439b-adfe-a2542d7b7c40 service nova] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:17.767838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa50bab-30fb-4b8b-a408-ae312fa83129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:17.768601 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa50bab-30fb-4b8b-a408-ae312fa83129 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:17.768881 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa50bab-30fb-4b8b-a408-ae312fa83129 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:17.769478 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa50bab-30fb-4b8b-a408-ae312fa83129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:17.771792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9fa50bab-30fb-4b8b-a408-ae312fa83129 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:17.859193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa50bab-30fb-4b8b-a408-ae312fa83129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:17.860626 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa50bab-30fb-4b8b-a408-ae312fa83129 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:17.870267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1107/2227] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:08:19.970751 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b636a9-9e07-4a97-b61d-4dde10b75b7e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:19.972620 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-76b636a9-9e07-4a97-b61d-4dde10b75b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:19.987486 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa50bab-30fb-4b8b-a408-ae312fa83129 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:20.000947 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa50bab-30fb-4b8b-a408-ae312fa83129 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:20.005044 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cafc9-08d2-40ed-877a-b60005827f41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:20.005683 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cafc9-08d2-40ed-877a-b60005827f41 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:20.005968 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cafc9-08d2-40ed-877a-b60005827f41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:20.006242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cafc9-08d2-40ed-877a-b60005827f41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:20.008050 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-693cafc9-08d2-40ed-877a-b60005827f41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:20.048553 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:20.061661 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b636a9-9e07-4a97-b61d-4dde10b75b7e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:20.066286 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6771e3-b464-48e8-8f8f-6910290fd7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:20.067023 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6771e3-b464-48e8-8f8f-6910290fd7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:20.067311 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6771e3-b464-48e8-8f8f-6910290fd7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:20.067594 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6771e3-b464-48e8-8f8f-6910290fd7a9 admin admin] PATH_INFO: `/v3/users/c90f4ac8f19a48509c6497720e97784b` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:20.068935 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6771e3-b464-48e8-8f8f-6910290fd7a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c90f4ac8f19a48509c6497720e97784b)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:20.071570 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6771e3-b464-48e8-8f8f-6910290fd7a9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:20.090228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693cafc9-08d2-40ed-877a-b60005827f41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:20.091748 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693cafc9-08d2-40ed-877a-b60005827f41 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:20.102526 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1108/2228] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:08:20.277537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ac6771e3-b464-48e8-8f8f-6910290fd7a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c90f4ac8f19a48509c6497720e97784b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:20.288277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cafc9-08d2-40ed-877a-b60005827f41 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:20.291948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-693cafc9-08d2-40ed-877a-b60005827f41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:20.293365 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ac6771e3-b464-48e8-8f8f-6910290fd7a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c90f4ac8f19a48509c6497720e97784b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:20.301946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ac6771e3-b464-48e8-8f8f-6910290fd7a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c90f4ac8f19a48509c6497720e97784b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:20.308033 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cafc9-08d2-40ed-877a-b60005827f41 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:20.313102 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d903e-37a4-4e76-9567-f454a2dccd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:20.313760 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d903e-37a4-4e76-9567-f454a2dccd28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:20.314071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d903e-37a4-4e76-9567-f454a2dccd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:20.314350 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d903e-37a4-4e76-9567-f454a2dccd28 admin admin] PATH_INFO: `/v3/users/a27bb57559b04fe686452b8c8d5ea041` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:20.315609 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d903e-37a4-4e76-9567-f454a2dccd28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a27bb57559b04fe686452b8c8d5ea041)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:20.318970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d903e-37a4-4e76-9567-f454a2dccd28 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:20.378386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ac6771e3-b464-48e8-8f8f-6910290fd7a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c90f4ac8f19a48509c6497720e97784b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:20.379056 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ac6771e3-b464-48e8-8f8f-6910290fd7a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c90f4ac8f19a48509c6497720e97784b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:20.385575 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1121/2229] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:08:19 2022] DELETE /identity/v3/users/c90f4ac8f19a48509c6497720e97784b => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:20.393978 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f651f8-c3eb-45dd-b6d7-60d7fb030628 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:20.394369 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f651f8-c3eb-45dd-b6d7-60d7fb030628 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:20.394663 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f651f8-c3eb-45dd-b6d7-60d7fb030628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:20.415749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-46f651f8-c3eb-45dd-b6d7-60d7fb030628 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:20.416972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-46f651f8-c3eb-45dd-b6d7-60d7fb030628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:20.464877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:20.489998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1122/2230] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:08:20.500621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f651f8-c3eb-45dd-b6d7-60d7fb030628 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:20.503199 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-46f651f8-c3eb-45dd-b6d7-60d7fb030628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:20.555546 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ca1d903e-37a4-4e76-9567-f454a2dccd28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a27bb57559b04fe686452b8c8d5ea041'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:20.566635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f651f8-c3eb-45dd-b6d7-60d7fb030628 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:20.570840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4524438-d4f3-4142-9047-45a16102a4d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:20.571696 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4524438-d4f3-4142-9047-45a16102a4d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:20.571916 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ca1d903e-37a4-4e76-9567-f454a2dccd28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a27bb57559b04fe686452b8c8d5ea041'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:20.572025 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4524438-d4f3-4142-9047-45a16102a4d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:20.572245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4524438-d4f3-4142-9047-45a16102a4d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:20.580868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ca1d903e-37a4-4e76-9567-f454a2dccd28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a27bb57559b04fe686452b8c8d5ea041'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:20.583075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4524438-d4f3-4142-9047-45a16102a4d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:20.585576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4524438-d4f3-4142-9047-45a16102a4d8 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:20.601388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1123/2231] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:08:20.701376 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ca1d903e-37a4-4e76-9567-f454a2dccd28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a27bb57559b04fe686452b8c8d5ea041'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:20.703509 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ca1d903e-37a4-4e76-9567-f454a2dccd28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a27bb57559b04fe686452b8c8d5ea041'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:20.715936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1109/2232] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:08:20 2022] DELETE /identity/v3/users/a27bb57559b04fe686452b8c8d5ea041 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:08:20.719568 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e87fa-7c9e-4397-84bb-3bfe5dc0aeb1 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:20.720358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e87fa-7c9e-4397-84bb-3bfe5dc0aeb1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:20.720742 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e87fa-7c9e-4397-84bb-3bfe5dc0aeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:20.744986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f32e87fa-7c9e-4397-84bb-3bfe5dc0aeb1 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:20.747657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f32e87fa-7c9e-4397-84bb-3bfe5dc0aeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:20.833195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1124/2233] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:08:20.840416 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d903e-37a4-4e76-9567-f454a2dccd28 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:20.841988 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ca1d903e-37a4-4e76-9567-f454a2dccd28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:21.013350 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d903e-37a4-4e76-9567-f454a2dccd28 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:21.026105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843b94a-5338-4bd9-a453-6e8a639aeaa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:21.030060 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843b94a-5338-4bd9-a453-6e8a639aeaa9 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:21.030431 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843b94a-5338-4bd9-a453-6e8a639aeaa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:21.030843 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843b94a-5338-4bd9-a453-6e8a639aeaa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:21.044768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843b94a-5338-4bd9-a453-6e8a639aeaa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:21.047050 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843b94a-5338-4bd9-a453-6e8a639aeaa9 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:21.058903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1110/2234] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:08:21.241186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e87fa-7c9e-4397-84bb-3bfe5dc0aeb1 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:21.249739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f32e87fa-7c9e-4397-84bb-3bfe5dc0aeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:21.266867 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e87fa-7c9e-4397-84bb-3bfe5dc0aeb1 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:21.280115 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940247f-59a4-4249-88a5-7f891d08f54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:21.280794 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940247f-59a4-4249-88a5-7f891d08f54e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:21.281215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940247f-59a4-4249-88a5-7f891d08f54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:21.281329 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940247f-59a4-4249-88a5-7f891d08f54e admin admin] PATH_INFO: `/v3/projects/ee462318179e41d2b72893d6936916f8` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:21.282881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2940247f-59a4-4249-88a5-7f891d08f54e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee462318179e41d2b72893d6936916f8)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:21.285285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2940247f-59a4-4249-88a5-7f891d08f54e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:21.460151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2940247f-59a4-4249-88a5-7f891d08f54e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee462318179e41d2b72893d6936916f8', 'request_id': 'req-2940247f-59a4-4249-88a5-7f891d08f54e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:21.522847 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2940247f-59a4-4249-88a5-7f891d08f54e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee462318179e41d2b72893d6936916f8', 'request_id': 'req-2940247f-59a4-4249-88a5-7f891d08f54e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:21.538156 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2940247f-59a4-4249-88a5-7f891d08f54e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee462318179e41d2b72893d6936916f8', 'request_id': 'req-2940247f-59a4-4249-88a5-7f891d08f54e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:21.555338 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2940247f-59a4-4249-88a5-7f891d08f54e admin admin] The token cache is being invalidate because project ee462318179e41d2b72893d6936916f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:08:21.555896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2940247f-59a4-4249-88a5-7f891d08f54e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:21.556731 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2940247f-59a4-4249-88a5-7f891d08f54e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:21.560554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1125/2235] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:08:21 2022] DELETE /identity/v3/projects/ee462318179e41d2b72893d6936916f8 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:21.565037 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6a170-a9e3-44e6-b02a-c542e76f9181 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:21.565592 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6a170-a9e3-44e6-b02a-c542e76f9181 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:21.566075 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6a170-a9e3-44e6-b02a-c542e76f9181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:21.590385 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-cbb6a170-a9e3-44e6-b02a-c542e76f9181 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:21.590385 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cbb6a170-a9e3-44e6-b02a-c542e76f9181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:21.645312 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:21.695103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1111/2236] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:08:21.695274 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ddb1b-ec18-4fdb-97e5-1875a1c68a92 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:21.695274 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ddb1b-ec18-4fdb-97e5-1875a1c68a92 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:21.695274 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ddb1b-ec18-4fdb-97e5-1875a1c68a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:21.724337 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6a170-a9e3-44e6-b02a-c542e76f9181 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:21.726306 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cbb6a170-a9e3-44e6-b02a-c542e76f9181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:21.738296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ac1ddb1b-ec18-4fdb-97e5-1875a1c68a92 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:21.746743 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ac1ddb1b-ec18-4fdb-97e5-1875a1c68a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:21.759002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6a170-a9e3-44e6-b02a-c542e76f9181 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:21.765814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8fa8f-7d83-4d56-89ac-b42d0f49d56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:21.766589 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8fa8f-7d83-4d56-89ac-b42d0f49d56a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:21.766941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8fa8f-7d83-4d56-89ac-b42d0f49d56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:21.767054 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1126/2237] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:08:21.767286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8fa8f-7d83-4d56-89ac-b42d0f49d56a admin admin] PATH_INFO: `/v3/projects/4dda01f4dfae4428b4ea737705d2edcd` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:21.768039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ddb1b-ec18-4fdb-97e5-1875a1c68a92 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:21.768953 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c8fa8f-7d83-4d56-89ac-b42d0f49d56a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dda01f4dfae4428b4ea737705d2edcd)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:21.771233 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c8fa8f-7d83-4d56-89ac-b42d0f49d56a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:21.774360 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:21.779476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ddb1b-ec18-4fdb-97e5-1875a1c68a92 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:21.784969 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dcc5f3-ba32-4445-a5da-09facf24e60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:21.786127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dcc5f3-ba32-4445-a5da-09facf24e60e admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:21.786790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dcc5f3-ba32-4445-a5da-09facf24e60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:21.787073 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dcc5f3-ba32-4445-a5da-09facf24e60e admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:21.788228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dcc5f3-ba32-4445-a5da-09facf24e60e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:08:21.788228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dcc5f3-ba32-4445-a5da-09facf24e60e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:21.789912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dcc5f3-ba32-4445-a5da-09facf24e60e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:21.809791 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1127/2238] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:08:21.814921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dcc5f3-ba32-4445-a5da-09facf24e60e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:21.825759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dcc5f3-ba32-4445-a5da-09facf24e60e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:21.842065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9bab5-6c6c-4309-8a2a-89e3c49f6ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:21.843843 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9bab5-6c6c-4309-8a2a-89e3c49f6ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:21.843843 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9bab5-6c6c-4309-8a2a-89e3c49f6ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:21.844355 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9bab5-6c6c-4309-8a2a-89e3c49f6ab6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:21.845931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9bab5-6c6c-4309-8a2a-89e3c49f6ab6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:21.849383 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9bab5-6c6c-4309-8a2a-89e3c49f6ab6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:21.987802 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1128/2239] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:08:21 2022] POST /identity/v3/projects => generated 373 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:08:22.001013 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9bab5-6c6c-4309-8a2a-89e3c49f6ab6 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:22.027563 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e3c8fa8f-7d83-4d56-89ac-b42d0f49d56a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dda01f4dfae4428b4ea737705d2edcd', 'request_id': 'req-e3c8fa8f-7d83-4d56-89ac-b42d0f49d56a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:22.115414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9bab5-6c6c-4309-8a2a-89e3c49f6ab6 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:22.124148 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01cf8c-f67e-4efa-aded-0c10b6862cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:22.124918 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01cf8c-f67e-4efa-aded-0c10b6862cf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:22.125275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01cf8c-f67e-4efa-aded-0c10b6862cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:22.125714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01cf8c-f67e-4efa-aded-0c10b6862cf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:22.126386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01cf8c-f67e-4efa-aded-0c10b6862cf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:22.128435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01cf8c-f67e-4efa-aded-0c10b6862cf0 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:22.147864 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e3c8fa8f-7d83-4d56-89ac-b42d0f49d56a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dda01f4dfae4428b4ea737705d2edcd', 'request_id': 'req-e3c8fa8f-7d83-4d56-89ac-b42d0f49d56a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:22.168403 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e3c8fa8f-7d83-4d56-89ac-b42d0f49d56a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dda01f4dfae4428b4ea737705d2edcd', 'request_id': 'req-e3c8fa8f-7d83-4d56-89ac-b42d0f49d56a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:22.190490 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e3c8fa8f-7d83-4d56-89ac-b42d0f49d56a admin admin] The token cache is being invalidate because project 4dda01f4dfae4428b4ea737705d2edcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:08:22.191190 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e3c8fa8f-7d83-4d56-89ac-b42d0f49d56a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:22.192268 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e3c8fa8f-7d83-4d56-89ac-b42d0f49d56a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:22.194392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1112/2240] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:08:21 2022] DELETE /identity/v3/projects/4dda01f4dfae4428b4ea737705d2edcd => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:08:22.203882 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366230ce-2bb5-494c-be53-2674b48b60c5 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:22.204188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366230ce-2bb5-494c-be53-2674b48b60c5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:22.204480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366230ce-2bb5-494c-be53-2674b48b60c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:22.233389 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-366230ce-2bb5-494c-be53-2674b48b60c5 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:22.233389 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-366230ce-2bb5-494c-be53-2674b48b60c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:22.242694 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1113/2241] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:22.251286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810a0ef-0c08-49ec-86b6-591fd4381295 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:22.251561 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810a0ef-0c08-49ec-86b6-591fd4381295 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:22.251839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810a0ef-0c08-49ec-86b6-591fd4381295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:22.276598 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b810a0ef-0c08-49ec-86b6-591fd4381295 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:22.277896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b810a0ef-0c08-49ec-86b6-591fd4381295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:22.296051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1114/2242] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:08:22.308064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810a0ef-0c08-49ec-86b6-591fd4381295 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:22.319931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810a0ef-0c08-49ec-86b6-591fd4381295 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:22.320403 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1129/2243] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:08:21 2022] POST /identity/v3/users => generated 287 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:08:22.322181 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01cf8c-f67e-4efa-aded-0c10b6862cf0 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:22.325672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7c01cf8c-f67e-4efa-aded-0c10b6862cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:22.326266 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da041e-03da-43e3-b90f-2947f8a0d31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:22.327501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da041e-03da-43e3-b90f-2947f8a0d31d admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:22.327783 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da041e-03da-43e3-b90f-2947f8a0d31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:22.328052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da041e-03da-43e3-b90f-2947f8a0d31d admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:22.328710 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97da041e-03da-43e3-b90f-2947f8a0d31d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:08:22.329154 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97da041e-03da-43e3-b90f-2947f8a0d31d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:22.330861 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97da041e-03da-43e3-b90f-2947f8a0d31d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:22.346677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1115/2244] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 12:08:22.346855 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01cf8c-f67e-4efa-aded-0c10b6862cf0 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:22.357726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da041e-03da-43e3-b90f-2947f8a0d31d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:22.360908 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117695e-8deb-4d79-b43d-72c857e7831a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:22.361553 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117695e-8deb-4d79-b43d-72c857e7831a admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:22.361844 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117695e-8deb-4d79-b43d-72c857e7831a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:22.362149 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117695e-8deb-4d79-b43d-72c857e7831a admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:22.362886 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117695e-8deb-4d79-b43d-72c857e7831a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:22.364514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117695e-8deb-4d79-b43d-72c857e7831a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:22.369352 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da041e-03da-43e3-b90f-2947f8a0d31d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:22.376252 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1130/2245] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:22.377104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4af92a-615a-4526-9806-6e3fbb70b162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:22.377738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4af92a-615a-4526-9806-6e3fbb70b162 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:22.378060 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4af92a-615a-4526-9806-6e3fbb70b162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:22.378376 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4af92a-615a-4526-9806-6e3fbb70b162 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:22.378972 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4af92a-615a-4526-9806-6e3fbb70b162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:22.380476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117695e-8deb-4d79-b43d-72c857e7831a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:22.381926 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4af92a-615a-4526-9806-6e3fbb70b162 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:22.393129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117695e-8deb-4d79-b43d-72c857e7831a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:22.400041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a75cb-ec78-40a2-a989-1351b661fda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:22.400792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a75cb-ec78-40a2-a989-1351b661fda5 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:22.401188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a75cb-ec78-40a2-a989-1351b661fda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:22.401595 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a75cb-ec78-40a2-a989-1351b661fda5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:22.402408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a75cb-ec78-40a2-a989-1351b661fda5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:22.404288 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a75cb-ec78-40a2-a989-1351b661fda5 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:22.416917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1131/2246] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:22.421477 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a75cb-ec78-40a2-a989-1351b661fda5 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:22.433723 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a75cb-ec78-40a2-a989-1351b661fda5 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:22.439346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85758ad-8024-4e90-88de-e0aad709d139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:22.440009 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85758ad-8024-4e90-88de-e0aad709d139 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:22.440316 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85758ad-8024-4e90-88de-e0aad709d139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:22.440596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85758ad-8024-4e90-88de-e0aad709d139 admin admin] PATH_INFO: `/v3/projects/7b000ea7ed224815adf2ac46bab168c3/users/557c77037aeb4465b1efa28d0d7112de/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:22.458499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85758ad-8024-4e90-88de-e0aad709d139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b000ea7ed224815adf2ac46bab168c3, user_id=557c77037aeb4465b1efa28d0d7112de, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:22.461189 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85758ad-8024-4e90-88de-e0aad709d139 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:22.505366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1116/2247] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:08:22 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:08:22.507094 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4af92a-615a-4526-9806-6e3fbb70b162 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:22.527506 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1132/2248] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:08:22 2022] PUT /identity/v3/projects/7b000ea7ed224815adf2ac46bab168c3/users/557c77037aeb4465b1efa28d0d7112de/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:08:22.538327 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85a5c3-1796-4edc-bbee-e3add9f38ac1 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:22.538635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85a5c3-1796-4edc-bbee-e3add9f38ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:22.538882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85a5c3-1796-4edc-bbee-e3add9f38ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:22.573203 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-db85a5c3-1796-4edc-bbee-e3add9f38ac1 None None] MFA Rules not processed for user `557c77037aeb4465b1efa28d0d7112de`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:22.574524 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-db85a5c3-1796-4edc-bbee-e3add9f38ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:22.581646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4af92a-615a-4526-9806-6e3fbb70b162 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:22.614668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:22.642305 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8aff1-8fb1-43ef-aee5-51406c863683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:22.642915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8aff1-8fb1-43ef-aee5-51406c863683 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:22.643208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8aff1-8fb1-43ef-aee5-51406c863683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:22.643478 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8aff1-8fb1-43ef-aee5-51406c863683 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:22.644119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e8aff1-8fb1-43ef-aee5-51406c863683 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:22.646138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e8aff1-8fb1-43ef-aee5-51406c863683 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:22.650405 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1133/2249] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:22.655082 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85a5c3-1796-4edc-bbee-e3add9f38ac1 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:22.671651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85a5c3-1796-4edc-bbee-e3add9f38ac1 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:22.676934 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bdd41-96fe-4272-8e23-b887750d8712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:22.677467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bdd41-96fe-4272-8e23-b887750d8712 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:22.677755 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bdd41-96fe-4272-8e23-b887750d8712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:22.678222 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bdd41-96fe-4272-8e23-b887750d8712 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:22.678878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1bdd41-96fe-4272-8e23-b887750d8712 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:22.680837 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1bdd41-96fe-4272-8e23-b887750d8712 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:22.737454 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1117/2250] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:08:22 2022] POST /identity/v3/users => generated 287 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:08:22.741525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8aff1-8fb1-43ef-aee5-51406c863683 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:22.752020 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8aff1-8fb1-43ef-aee5-51406c863683 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:22.756881 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c46541-084d-4087-957d-096c9a3b8066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:22.757499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c46541-084d-4087-957d-096c9a3b8066 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:22.757783 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c46541-084d-4087-957d-096c9a3b8066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:22.758098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c46541-084d-4087-957d-096c9a3b8066 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:22.758741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c46541-084d-4087-957d-096c9a3b8066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:22.760372 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c46541-084d-4087-957d-096c9a3b8066 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:22.771260 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1118/2251] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:22.775502 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c46541-084d-4087-957d-096c9a3b8066 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:22.786292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c46541-084d-4087-957d-096c9a3b8066 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:22.792726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352e958-8472-4b18-b790-502ff04de1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:22.793881 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352e958-8472-4b18-b790-502ff04de1cf admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:22.793881 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352e958-8472-4b18-b790-502ff04de1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:22.797424 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352e958-8472-4b18-b790-502ff04de1cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:22.797424 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352e958-8472-4b18-b790-502ff04de1cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:22.797424 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352e958-8472-4b18-b790-502ff04de1cf admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:22.806665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1134/2252] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:08:22 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:08:22.808498 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1119/2253] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:08:22.809929 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bdd41-96fe-4272-8e23-b887750d8712 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:22.815915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352e958-8472-4b18-b790-502ff04de1cf admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:22.827971 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352e958-8472-4b18-b790-502ff04de1cf admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:22.845558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:22.945675 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bdd41-96fe-4272-8e23-b887750d8712 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:22.949252 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ab506-a6fd-4ef8-8686-87c089d92ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:22.950440 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ab506-a6fd-4ef8-8686-87c089d92ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:22.950440 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ab506-a6fd-4ef8-8686-87c089d92ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:22.950681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ab506-a6fd-4ef8-8686-87c089d92ddf admin admin] PATH_INFO: `/v3/projects/2e692d812c3842608e10769277212d85/users/e133fb1749004c36a38ac028782294e2/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:22.955490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddc822-ab74-4bbb-8fce-99e83996e805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:22.956964 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddc822-ab74-4bbb-8fce-99e83996e805 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:22.957423 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddc822-ab74-4bbb-8fce-99e83996e805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:22.958554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddc822-ab74-4bbb-8fce-99e83996e805 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:22.958658 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddc822-ab74-4bbb-8fce-99e83996e805 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:22.960946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddc822-ab74-4bbb-8fce-99e83996e805 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:22.977119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ab506-a6fd-4ef8-8686-87c089d92ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e692d812c3842608e10769277212d85, user_id=e133fb1749004c36a38ac028782294e2, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:22.980872 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ab506-a6fd-4ef8-8686-87c089d92ddf admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:23.093238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1135/2254] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:08:22 2022] POST /identity/v3/users => generated 294 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:08:23.097301 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddc822-ab74-4bbb-8fce-99e83996e805 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:23.108331 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddc822-ab74-4bbb-8fce-99e83996e805 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:23.113586 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b4bd4-1224-47c4-bbe5-6f725f692b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:23.114287 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b4bd4-1224-47c4-bbe5-6f725f692b96 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:23.114573 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b4bd4-1224-47c4-bbe5-6f725f692b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:23.114881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b4bd4-1224-47c4-bbe5-6f725f692b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:23.115632 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b4bd4-1224-47c4-bbe5-6f725f692b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:23.118354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b4bd4-1224-47c4-bbe5-6f725f692b96 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:23.134499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1136/2255] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:08:23.140152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b4bd4-1224-47c4-bbe5-6f725f692b96 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:23.141152 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1120/2256] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:08:22 2022] PUT /identity/v3/projects/2e692d812c3842608e10769277212d85/users/e133fb1749004c36a38ac028782294e2/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:23.146067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c03697-d566-4d31-9145-ecc6b95c33a4 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:23.146530 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c03697-d566-4d31-9145-ecc6b95c33a4 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:23.146911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c03697-d566-4d31-9145-ecc6b95c33a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:23.157403 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b4bd4-1224-47c4-bbe5-6f725f692b96 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:23.212416 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-74c03697-d566-4d31-9145-ecc6b95c33a4 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:23.214658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-74c03697-d566-4d31-9145-ecc6b95c33a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:23.297132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9dcff-f58f-478b-a462-630a16630ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:23.298290 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9dcff-f58f-478b-a462-630a16630ff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:23.298779 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9dcff-f58f-478b-a462-630a16630ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:23.299768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9dcff-f58f-478b-a462-630a16630ff9 admin admin] PATH_INFO: `/v3/projects/8c14b4b05b8d44f8837bf964784ec2a2/users/4584d5a6ba0c431cb7daad16154897f7/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:23.326980 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9dcff-f58f-478b-a462-630a16630ff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c14b4b05b8d44f8837bf964784ec2a2, user_id=4584d5a6ba0c431cb7daad16154897f7, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:23.331619 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9dcff-f58f-478b-a462-630a16630ff9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:23.333204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1121/2257] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:23.346193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c03697-d566-4d31-9145-ecc6b95c33a4 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:23.347900 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-74c03697-d566-4d31-9145-ecc6b95c33a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:23.428110 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c03697-d566-4d31-9145-ecc6b95c33a4 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:23.438525 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1137/2258] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:08:23 2022] PUT /identity/v3/projects/8c14b4b05b8d44f8837bf964784ec2a2/users/4584d5a6ba0c431cb7daad16154897f7/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:08:23.442983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9dcff-f58f-478b-a462-630a16630ff9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:23.448835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:23.503676 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e15ca-d553-42e7-a09c-195f562ee426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:23.504376 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e15ca-d553-42e7-a09c-195f562ee426 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:23.504661 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e15ca-d553-42e7-a09c-195f562ee426 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:23.505539 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e15ca-d553-42e7-a09c-195f562ee426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:23.517275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:23.537845 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9dcff-f58f-478b-a462-630a16630ff9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:23.542887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d5d39-f51e-4d40-8aa0-97c2632ecf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:23.543602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d5d39-f51e-4d40-8aa0-97c2632ecf53 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:23.543915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d5d39-f51e-4d40-8aa0-97c2632ecf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:23.544188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d5d39-f51e-4d40-8aa0-97c2632ecf53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:23.545014 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d5d39-f51e-4d40-8aa0-97c2632ecf53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:23.546721 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d5d39-f51e-4d40-8aa0-97c2632ecf53 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:23.558594 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1138/2259] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:08:23.563069 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d5d39-f51e-4d40-8aa0-97c2632ecf53 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:23.573845 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d5d39-f51e-4d40-8aa0-97c2632ecf53 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:23.585520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41607d1-8306-4e11-86fd-c94e6e3c91a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:23.586234 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41607d1-8306-4e11-86fd-c94e6e3c91a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:23.586515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41607d1-8306-4e11-86fd-c94e6e3c91a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:23.586794 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41607d1-8306-4e11-86fd-c94e6e3c91a1 admin admin] PATH_INFO: `/v3/projects/8c14b4b05b8d44f8837bf964784ec2a2/users/4584d5a6ba0c431cb7daad16154897f7/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:23.588839 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41607d1-8306-4e11-86fd-c94e6e3c91a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c14b4b05b8d44f8837bf964784ec2a2, user_id=4584d5a6ba0c431cb7daad16154897f7, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:23.591754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41607d1-8306-4e11-86fd-c94e6e3c91a1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:23.600105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e15ca-d553-42e7-a09c-195f562ee426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:23.601569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e15ca-d553-42e7-a09c-195f562ee426 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:23.610075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1139/2260] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:08:23 2022] PUT /identity/v3/projects/8c14b4b05b8d44f8837bf964784ec2a2/users/4584d5a6ba0c431cb7daad16154897f7/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:08:23.613230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1122/2261] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:08:23.618992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc30a7-b854-4085-a80c-8785a7452cd2 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:23.621973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc30a7-b854-4085-a80c-8785a7452cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:23.621973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc30a7-b854-4085-a80c-8785a7452cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:23.659266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-68cc30a7-b854-4085-a80c-8785a7452cd2 None None] MFA Rules not processed for user `4584d5a6ba0c431cb7daad16154897f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:23.660671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-68cc30a7-b854-4085-a80c-8785a7452cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:23.741066 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1140/2262] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:23.753203 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e15ca-d553-42e7-a09c-195f562ee426 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:23.755137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d89e15ca-d553-42e7-a09c-195f562ee426 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:23.853584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e15ca-d553-42e7-a09c-195f562ee426 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:23.892760 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:23.909351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2b9f1-bf54-40e2-8996-a301ee897afa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:23.910007 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2b9f1-bf54-40e2-8996-a301ee897afa service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:23.910294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2b9f1-bf54-40e2-8996-a301ee897afa service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:23.910572 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2b9f1-bf54-40e2-8996-a301ee897afa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:23.975529 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2b9f1-bf54-40e2-8996-a301ee897afa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:23.977112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2b9f1-bf54-40e2-8996-a301ee897afa service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:23.987654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1123/2263] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:08:25.294057 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cd370-8aac-41b5-8367-0f5aac4afc93 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:25.294057 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cd370-8aac-41b5-8367-0f5aac4afc93 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:25.294057 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cd370-8aac-41b5-8367-0f5aac4afc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:25.342619 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-c68cd370-8aac-41b5-8367-0f5aac4afc93 None None] MFA Rules not processed for user `e133fb1749004c36a38ac028782294e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:25.344143 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c68cd370-8aac-41b5-8367-0f5aac4afc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:25.430856 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1141/2264] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:25.443568 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2b9f1-bf54-40e2-8996-a301ee897afa service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:25.444996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cfb2b9f1-bf54-40e2-8996-a301ee897afa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:25.456123 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2b9f1-bf54-40e2-8996-a301ee897afa service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:25.460006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd52c654-3ffa-40ce-8f7a-4ce600094a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:25.460603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52c654-3ffa-40ce-8f7a-4ce600094a64 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:25.460911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52c654-3ffa-40ce-8f7a-4ce600094a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:25.461187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52c654-3ffa-40ce-8f7a-4ce600094a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:25.473348 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd52c654-3ffa-40ce-8f7a-4ce600094a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:25.477215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd52c654-3ffa-40ce-8f7a-4ce600094a64 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:25.498982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1124/2265] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:08:27.543671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cd370-8aac-41b5-8367-0f5aac4afc93 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:27.545165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c68cd370-8aac-41b5-8367-0f5aac4afc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:27.600139 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:27.624852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cd370-8aac-41b5-8367-0f5aac4afc93 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:27.631051 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2887bd8-05ed-448c-93a9-74f24b1dedc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:27.631840 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2887bd8-05ed-448c-93a9-74f24b1dedc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:27.632162 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2887bd8-05ed-448c-93a9-74f24b1dedc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:27.632497 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2887bd8-05ed-448c-93a9-74f24b1dedc2 admin admin] PATH_INFO: `/v3/users/a13106affcb943a8a42b2f1edb3143bb` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:27.633880 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2887bd8-05ed-448c-93a9-74f24b1dedc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a13106affcb943a8a42b2f1edb3143bb)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:27.636003 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2887bd8-05ed-448c-93a9-74f24b1dedc2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:27.814186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d2887bd8-05ed-448c-93a9-74f24b1dedc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a13106affcb943a8a42b2f1edb3143bb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:27.827967 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d2887bd8-05ed-448c-93a9-74f24b1dedc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a13106affcb943a8a42b2f1edb3143bb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:27.837640 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d2887bd8-05ed-448c-93a9-74f24b1dedc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a13106affcb943a8a42b2f1edb3143bb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:27.926950 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d2887bd8-05ed-448c-93a9-74f24b1dedc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a13106affcb943a8a42b2f1edb3143bb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:27.929601 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d2887bd8-05ed-448c-93a9-74f24b1dedc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a13106affcb943a8a42b2f1edb3143bb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:27.938977 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1142/2266] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:08:27 2022] DELETE /identity/v3/users/a13106affcb943a8a42b2f1edb3143bb => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:27.943554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7733c-2985-49fc-abaa-f850e48b8e97 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:27.944040 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7733c-2985-49fc-abaa-f850e48b8e97 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:27.944135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7733c-2985-49fc-abaa-f850e48b8e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:27.963393 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b0f7733c-2985-49fc-abaa-f850e48b8e97 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:27.964673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b0f7733c-2985-49fc-abaa-f850e48b8e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:28.037961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1125/2267] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:28.049848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2887bd8-05ed-448c-93a9-74f24b1dedc2 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:28.051430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d2887bd8-05ed-448c-93a9-74f24b1dedc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:28.118694 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2887bd8-05ed-448c-93a9-74f24b1dedc2 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:28.122948 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1773488-2f3c-4db0-8f9c-541ecbc1b4ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:28.123564 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1773488-2f3c-4db0-8f9c-541ecbc1b4ad service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:28.124423 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1773488-2f3c-4db0-8f9c-541ecbc1b4ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:28.124706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1773488-2f3c-4db0-8f9c-541ecbc1b4ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:28.134624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1773488-2f3c-4db0-8f9c-541ecbc1b4ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:28.136092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1773488-2f3c-4db0-8f9c-541ecbc1b4ad service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:28.148692 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1143/2268] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:08:28.806577 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7733c-2985-49fc-abaa-f850e48b8e97 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:28.808517 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b0f7733c-2985-49fc-abaa-f850e48b8e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:28.820871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7733c-2985-49fc-abaa-f850e48b8e97 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:28.834014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f585ff5-6e93-47fd-9ff1-88a39b315fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:28.834889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f585ff5-6e93-47fd-9ff1-88a39b315fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:28.835348 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f585ff5-6e93-47fd-9ff1-88a39b315fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:28.835765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f585ff5-6e93-47fd-9ff1-88a39b315fca admin admin] PATH_INFO: `/v3/projects/ca512c2e96e64742a6426b2990956b99` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:28.837508 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f585ff5-6e93-47fd-9ff1-88a39b315fca admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca512c2e96e64742a6426b2990956b99)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:28.841119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f585ff5-6e93-47fd-9ff1-88a39b315fca admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:28.981981 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3f585ff5-6e93-47fd-9ff1-88a39b315fca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca512c2e96e64742a6426b2990956b99', 'request_id': 'req-3f585ff5-6e93-47fd-9ff1-88a39b315fca'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:29.057939 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3f585ff5-6e93-47fd-9ff1-88a39b315fca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca512c2e96e64742a6426b2990956b99', 'request_id': 'req-3f585ff5-6e93-47fd-9ff1-88a39b315fca'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:29.064579 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3f585ff5-6e93-47fd-9ff1-88a39b315fca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca512c2e96e64742a6426b2990956b99', 'request_id': 'req-3f585ff5-6e93-47fd-9ff1-88a39b315fca'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:29.072764 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:29.076666 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3f585ff5-6e93-47fd-9ff1-88a39b315fca admin admin] The token cache is being invalidate because project ca512c2e96e64742a6426b2990956b99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:08:29.076958 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3f585ff5-6e93-47fd-9ff1-88a39b315fca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:29.077830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3f585ff5-6e93-47fd-9ff1-88a39b315fca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:29.079605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1126/2269] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:08:28 2022] DELETE /identity/v3/projects/ca512c2e96e64742a6426b2990956b99 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:08:29.083582 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1773488-2f3c-4db0-8f9c-541ecbc1b4ad service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:29.097339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1773488-2f3c-4db0-8f9c-541ecbc1b4ad service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:29.159199 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769acc3-2ac9-4637-8097-09c3bd6f71e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:29.159835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769acc3-2ac9-4637-8097-09c3bd6f71e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:29.160121 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769acc3-2ac9-4637-8097-09c3bd6f71e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:29.160395 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769acc3-2ac9-4637-8097-09c3bd6f71e2 admin admin] PATH_INFO: `/v3/users/1c5f791b1e2b4633b3ccb66cd304850f` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:29.161516 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769acc3-2ac9-4637-8097-09c3bd6f71e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c5f791b1e2b4633b3ccb66cd304850f)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:29.163446 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769acc3-2ac9-4637-8097-09c3bd6f71e2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:29.166045 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:29.319852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a769acc3-2ac9-4637-8097-09c3bd6f71e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c5f791b1e2b4633b3ccb66cd304850f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:29.333410 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a769acc3-2ac9-4637-8097-09c3bd6f71e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c5f791b1e2b4633b3ccb66cd304850f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:29.341310 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a769acc3-2ac9-4637-8097-09c3bd6f71e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c5f791b1e2b4633b3ccb66cd304850f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:29.429454 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a769acc3-2ac9-4637-8097-09c3bd6f71e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c5f791b1e2b4633b3ccb66cd304850f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:29.430676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a769acc3-2ac9-4637-8097-09c3bd6f71e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c5f791b1e2b4633b3ccb66cd304850f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:29.438078 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1144/2270] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:08:29 2022] DELETE /identity/v3/users/1c5f791b1e2b4633b3ccb66cd304850f => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:29.574210 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80e893-2098-423c-980a-cd3a08b0170e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:29.574830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80e893-2098-423c-980a-cd3a08b0170e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:29.575683 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80e893-2098-423c-980a-cd3a08b0170e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:29.598740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ef80e893-2098-423c-980a-cd3a08b0170e None None] MFA Rules not processed for user `e133fb1749004c36a38ac028782294e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:29.599835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ef80e893-2098-423c-980a-cd3a08b0170e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:29.676271 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1127/2271] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:30.068560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769acc3-2ac9-4637-8097-09c3bd6f71e2 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:30.073835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a769acc3-2ac9-4637-8097-09c3bd6f71e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:30.164315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769acc3-2ac9-4637-8097-09c3bd6f71e2 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:30.173305 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6985dac-c029-4485-b81b-42077aa792aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:30.183781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6985dac-c029-4485-b81b-42077aa792aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:30.183781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6985dac-c029-4485-b81b-42077aa792aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:30.183781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6985dac-c029-4485-b81b-42077aa792aa admin admin] PATH_INFO: `/v3/projects/d070ecb201ca4812a9a7441ae564596f` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:30.183781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6985dac-c029-4485-b81b-42077aa792aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d070ecb201ca4812a9a7441ae564596f)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:30.183781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6985dac-c029-4485-b81b-42077aa792aa admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:30.398803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a6985dac-c029-4485-b81b-42077aa792aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd070ecb201ca4812a9a7441ae564596f', 'request_id': 'req-a6985dac-c029-4485-b81b-42077aa792aa'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:30.498130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a6985dac-c029-4485-b81b-42077aa792aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd070ecb201ca4812a9a7441ae564596f', 'request_id': 'req-a6985dac-c029-4485-b81b-42077aa792aa'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:30.505895 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a6985dac-c029-4485-b81b-42077aa792aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd070ecb201ca4812a9a7441ae564596f', 'request_id': 'req-a6985dac-c029-4485-b81b-42077aa792aa'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:30.519685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a6985dac-c029-4485-b81b-42077aa792aa admin admin] The token cache is being invalidate because project d070ecb201ca4812a9a7441ae564596f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:08:30.520076 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a6985dac-c029-4485-b81b-42077aa792aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:30.520833 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a6985dac-c029-4485-b81b-42077aa792aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:30.522724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1145/2272] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:08:30 2022] DELETE /identity/v3/projects/d070ecb201ca4812a9a7441ae564596f => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:30.527106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80e893-2098-423c-980a-cd3a08b0170e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:30.539600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80e893-2098-423c-980a-cd3a08b0170e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:30.605206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5d9cf-f45f-41de-a9ce-65149b2c36d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:30.605885 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5d9cf-f45f-41de-a9ce-65149b2c36d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:30.606173 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5d9cf-f45f-41de-a9ce-65149b2c36d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:30.606454 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5d9cf-f45f-41de-a9ce-65149b2c36d4 admin admin] PATH_INFO: `/v3/users/0afebe6ff72440b48d7d999ace053e3a` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:30.607611 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda5d9cf-f45f-41de-a9ce-65149b2c36d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0afebe6ff72440b48d7d999ace053e3a)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:30.609806 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda5d9cf-f45f-41de-a9ce-65149b2c36d4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:30.613464 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6985dac-c029-4485-b81b-42077aa792aa admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:30.615143 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a6985dac-c029-4485-b81b-42077aa792aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:30.632948 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6985dac-c029-4485-b81b-42077aa792aa admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:30.639208 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510889c-da2c-4f79-9ed2-817a67279858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:30.640278 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510889c-da2c-4f79-9ed2-817a67279858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:30.640602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510889c-da2c-4f79-9ed2-817a67279858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:30.640867 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510889c-da2c-4f79-9ed2-817a67279858 admin admin] PATH_INFO: `/v3/users/557c77037aeb4465b1efa28d0d7112de` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:30.642100 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510889c-da2c-4f79-9ed2-817a67279858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=557c77037aeb4465b1efa28d0d7112de)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:30.644114 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510889c-da2c-4f79-9ed2-817a67279858 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:30.650359 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:30.849940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-dda5d9cf-f45f-41de-a9ce-65149b2c36d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0afebe6ff72440b48d7d999ace053e3a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:30.866463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-dda5d9cf-f45f-41de-a9ce-65149b2c36d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0afebe6ff72440b48d7d999ace053e3a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:30.879410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-dda5d9cf-f45f-41de-a9ce-65149b2c36d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0afebe6ff72440b48d7d999ace053e3a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:30.880396 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6510889c-da2c-4f79-9ed2-817a67279858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '557c77037aeb4465b1efa28d0d7112de'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:30.894961 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6510889c-da2c-4f79-9ed2-817a67279858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '557c77037aeb4465b1efa28d0d7112de'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:30.908250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6510889c-da2c-4f79-9ed2-817a67279858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '557c77037aeb4465b1efa28d0d7112de'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:30.982433 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-dda5d9cf-f45f-41de-a9ce-65149b2c36d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0afebe6ff72440b48d7d999ace053e3a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:30.983104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-dda5d9cf-f45f-41de-a9ce-65149b2c36d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0afebe6ff72440b48d7d999ace053e3a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:30.990691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1128/2273] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:08:30 2022] DELETE /identity/v3/users/0afebe6ff72440b48d7d999ace053e3a => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:31.070016 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6510889c-da2c-4f79-9ed2-817a67279858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '557c77037aeb4465b1efa28d0d7112de'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:31.070863 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6510889c-da2c-4f79-9ed2-817a67279858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '557c77037aeb4465b1efa28d0d7112de'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:31.078866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1146/2274] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:08:30 2022] DELETE /identity/v3/users/557c77037aeb4465b1efa28d0d7112de => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:31.083674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e448053-4f2b-44a0-973c-3d727a8c36d7 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:31.084212 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e448053-4f2b-44a0-973c-3d727a8c36d7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:31.084672 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e448053-4f2b-44a0-973c-3d727a8c36d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:31.106280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8e448053-4f2b-44a0-973c-3d727a8c36d7 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:31.107521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8e448053-4f2b-44a0-973c-3d727a8c36d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:31.134508 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:31.187462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1129/2275] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:31.201176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510889c-da2c-4f79-9ed2-817a67279858 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:31.204163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6510889c-da2c-4f79-9ed2-817a67279858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:31.309763 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510889c-da2c-4f79-9ed2-817a67279858 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:31.315403 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24384f65-ce0e-420d-9cbe-ea980a2607aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:31.316452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24384f65-ce0e-420d-9cbe-ea980a2607aa service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:31.316976 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24384f65-ce0e-420d-9cbe-ea980a2607aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:31.317362 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24384f65-ce0e-420d-9cbe-ea980a2607aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:31.331033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24384f65-ce0e-420d-9cbe-ea980a2607aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:31.332493 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24384f65-ce0e-420d-9cbe-ea980a2607aa service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:31.344599 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1147/2276] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:08:31.520135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e448053-4f2b-44a0-973c-3d727a8c36d7 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:31.522973 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8e448053-4f2b-44a0-973c-3d727a8c36d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:31.547713 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e448053-4f2b-44a0-973c-3d727a8c36d7 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:31.556334 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb64bf-de23-4025-af60-dc3103918b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:31.558158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb64bf-de23-4025-af60-dc3103918b13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:31.558664 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb64bf-de23-4025-af60-dc3103918b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:31.559265 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb64bf-de23-4025-af60-dc3103918b13 admin admin] PATH_INFO: `/v3/projects/30f508d051884ca299571768f1f6d8b6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:31.574240 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb64bf-de23-4025-af60-dc3103918b13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30f508d051884ca299571768f1f6d8b6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:31.578469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb64bf-de23-4025-af60-dc3103918b13 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:31.798567 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5eeb64bf-de23-4025-af60-dc3103918b13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30f508d051884ca299571768f1f6d8b6', 'request_id': 'req-5eeb64bf-de23-4025-af60-dc3103918b13'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:31.915130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5eeb64bf-de23-4025-af60-dc3103918b13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30f508d051884ca299571768f1f6d8b6', 'request_id': 'req-5eeb64bf-de23-4025-af60-dc3103918b13'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:31.923320 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5eeb64bf-de23-4025-af60-dc3103918b13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30f508d051884ca299571768f1f6d8b6', 'request_id': 'req-5eeb64bf-de23-4025-af60-dc3103918b13'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:31.942908 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5eeb64bf-de23-4025-af60-dc3103918b13 admin admin] The token cache is being invalidate because project 30f508d051884ca299571768f1f6d8b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:08:31.943721 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5eeb64bf-de23-4025-af60-dc3103918b13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:31.944034 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5eeb64bf-de23-4025-af60-dc3103918b13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:31.945696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1130/2277] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:08:31 2022] DELETE /identity/v3/projects/30f508d051884ca299571768f1f6d8b6 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:31.950547 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aba247-f252-4675-8919-ace82f72512b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:31.953657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aba247-f252-4675-8919-ace82f72512b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:31.953657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aba247-f252-4675-8919-ace82f72512b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:31.973160 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-50aba247-f252-4675-8919-ace82f72512b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:31.974552 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-50aba247-f252-4675-8919-ace82f72512b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:32.049985 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1148/2278] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:32.057699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f071d4-d719-4d13-b25c-7be43370e724 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:32.058047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f071d4-d719-4d13-b25c-7be43370e724 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:32.058333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f071d4-d719-4d13-b25c-7be43370e724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:32.091161 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-00f071d4-d719-4d13-b25c-7be43370e724 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:32.093814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-00f071d4-d719-4d13-b25c-7be43370e724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:32.105198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1131/2279] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:08:32.110917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aba247-f252-4675-8919-ace82f72512b None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:32.118637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f071d4-d719-4d13-b25c-7be43370e724 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:32.128186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aba247-f252-4675-8919-ace82f72512b None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:32.133685 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-00f071d4-d719-4d13-b25c-7be43370e724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:32.142045 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed4c19-9721-4a59-8073-7833acb27a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:32.143146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed4c19-9721-4a59-8073-7833acb27a29 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:32.143666 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed4c19-9721-4a59-8073-7833acb27a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:32.144126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed4c19-9721-4a59-8073-7833acb27a29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:32.144923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed4c19-9721-4a59-8073-7833acb27a29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:08:32.151338 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed4c19-9721-4a59-8073-7833acb27a29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:32.151338 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed4c19-9721-4a59-8073-7833acb27a29 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:32.154025 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f071d4-d719-4d13-b25c-7be43370e724 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:32.160782 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965807a-947e-424d-a344-1f0da2f42f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:32.161609 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965807a-947e-424d-a344-1f0da2f42f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:32.162091 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965807a-947e-424d-a344-1f0da2f42f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:32.166859 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965807a-947e-424d-a344-1f0da2f42f0d admin admin] PATH_INFO: `/v3/projects/7b000ea7ed224815adf2ac46bab168c3` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:32.168331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1965807a-947e-424d-a344-1f0da2f42f0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b000ea7ed224815adf2ac46bab168c3)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:32.170401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1965807a-947e-424d-a344-1f0da2f42f0d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:32.176368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1149/2280] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:08:32.183160 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed4c19-9721-4a59-8073-7833acb27a29 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:32.195612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed4c19-9721-4a59-8073-7833acb27a29 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:32.203270 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253ce57-499d-4f4d-8ce8-d5ea201bb189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:32.205321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253ce57-499d-4f4d-8ce8-d5ea201bb189 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:32.205852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253ce57-499d-4f4d-8ce8-d5ea201bb189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:32.206161 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253ce57-499d-4f4d-8ce8-d5ea201bb189 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:32.207251 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253ce57-499d-4f4d-8ce8-d5ea201bb189 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:32.210366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253ce57-499d-4f4d-8ce8-d5ea201bb189 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:32.223523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:32.331275 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1965807a-947e-424d-a344-1f0da2f42f0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b000ea7ed224815adf2ac46bab168c3', 'request_id': 'req-1965807a-947e-424d-a344-1f0da2f42f0d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:32.356623 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1150/2281] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:08:32 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:08:32.361578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253ce57-499d-4f4d-8ce8-d5ea201bb189 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:32.430463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253ce57-499d-4f4d-8ce8-d5ea201bb189 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:32.435981 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a005e5-c158-4052-aff0-189ae02f8a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:32.437145 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a005e5-c158-4052-aff0-189ae02f8a8b admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:32.437428 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a005e5-c158-4052-aff0-189ae02f8a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:32.437708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a005e5-c158-4052-aff0-189ae02f8a8b admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:32.438400 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a005e5-c158-4052-aff0-189ae02f8a8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:32.440377 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a005e5-c158-4052-aff0-189ae02f8a8b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:32.452393 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1965807a-947e-424d-a344-1f0da2f42f0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b000ea7ed224815adf2ac46bab168c3', 'request_id': 'req-1965807a-947e-424d-a344-1f0da2f42f0d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:32.467267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1965807a-947e-424d-a344-1f0da2f42f0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b000ea7ed224815adf2ac46bab168c3', 'request_id': 'req-1965807a-947e-424d-a344-1f0da2f42f0d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:32.484315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1965807a-947e-424d-a344-1f0da2f42f0d admin admin] The token cache is being invalidate because project 7b000ea7ed224815adf2ac46bab168c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:08:32.485093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1965807a-947e-424d-a344-1f0da2f42f0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:32.487522 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1965807a-947e-424d-a344-1f0da2f42f0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:32.488745 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1132/2282] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:08:32 2022] DELETE /identity/v3/projects/7b000ea7ed224815adf2ac46bab168c3 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:08:32.492968 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965807a-947e-424d-a344-1f0da2f42f0d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:32.494570 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1965807a-947e-424d-a344-1f0da2f42f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:32.524474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:32.555589 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1151/2283] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:08:32 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:08:32.556834 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a005e5-c158-4052-aff0-189ae02f8a8b admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:32.558881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e2a005e5-c158-4052-aff0-189ae02f8a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:32.566572 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965807a-947e-424d-a344-1f0da2f42f0d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:32.571511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5fe72d-e2e4-4b07-8fe4-916807611cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:32.573448 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fe72d-e2e4-4b07-8fe4-916807611cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:32.573448 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fe72d-e2e4-4b07-8fe4-916807611cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:32.573448 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fe72d-e2e4-4b07-8fe4-916807611cb9 admin admin] PATH_INFO: `/v3/users/4584d5a6ba0c431cb7daad16154897f7` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:32.576130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a005e5-c158-4052-aff0-189ae02f8a8b admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:32.576227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5fe72d-e2e4-4b07-8fe4-916807611cb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4584d5a6ba0c431cb7daad16154897f7)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:32.576227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5fe72d-e2e4-4b07-8fe4-916807611cb9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:32.584667 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc0067-c8c9-41ae-92ad-59b352f0cf70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:32.584667 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc0067-c8c9-41ae-92ad-59b352f0cf70 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:32.584667 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc0067-c8c9-41ae-92ad-59b352f0cf70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:32.584667 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc0067-c8c9-41ae-92ad-59b352f0cf70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:32.585117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc0067-c8c9-41ae-92ad-59b352f0cf70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:32.585117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc0067-c8c9-41ae-92ad-59b352f0cf70 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:32.593568 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1152/2284] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:08:32.597838 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc0067-c8c9-41ae-92ad-59b352f0cf70 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:32.610564 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc0067-c8c9-41ae-92ad-59b352f0cf70 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:32.615679 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1f8c0-a079-4b6d-baab-9ffe101f38e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:32.616307 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1f8c0-a079-4b6d-baab-9ffe101f38e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:32.616687 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1f8c0-a079-4b6d-baab-9ffe101f38e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:32.616963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1f8c0-a079-4b6d-baab-9ffe101f38e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:32.617607 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1f8c0-a079-4b6d-baab-9ffe101f38e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:32.619302 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1f8c0-a079-4b6d-baab-9ffe101f38e3 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:32.630756 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1153/2285] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:32.635013 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1f8c0-a079-4b6d-baab-9ffe101f38e3 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:32.645086 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1f8c0-a079-4b6d-baab-9ffe101f38e3 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:32.654689 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa021f9-8787-4fe9-8330-c28191ecbbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:32.656173 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa021f9-8787-4fe9-8330-c28191ecbbde admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:32.656445 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa021f9-8787-4fe9-8330-c28191ecbbde admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:32.656898 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa021f9-8787-4fe9-8330-c28191ecbbde admin admin] PATH_INFO: `/v3/projects/1488f00baa934f2e9b910644a179c755/users/8fe575351bdc456682db592f0c723ea6/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:32.680617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa021f9-8787-4fe9-8330-c28191ecbbde admin admin] RBAC: Authorizing `identity:create_grant(project_id=1488f00baa934f2e9b910644a179c755, user_id=8fe575351bdc456682db592f0c723ea6, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:32.683318 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa021f9-8787-4fe9-8330-c28191ecbbde admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:32.831511 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1154/2286] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:08:32 2022] PUT /identity/v3/projects/1488f00baa934f2e9b910644a179c755/users/8fe575351bdc456682db592f0c723ea6/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:08:32.840171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264123be-9080-4597-a0c1-b1d5ef46c1d4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:32.840627 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264123be-9080-4597-a0c1-b1d5ef46c1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:32.841031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264123be-9080-4597-a0c1-b1d5ef46c1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:32.879488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1b5fe72d-e2e4-4b07-8fe4-916807611cb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4584d5a6ba0c431cb7daad16154897f7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:32.891912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-264123be-9080-4597-a0c1-b1d5ef46c1d4 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:32.893756 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-264123be-9080-4597-a0c1-b1d5ef46c1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:32.908071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1b5fe72d-e2e4-4b07-8fe4-916807611cb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4584d5a6ba0c431cb7daad16154897f7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:32.931268 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1b5fe72d-e2e4-4b07-8fe4-916807611cb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4584d5a6ba0c431cb7daad16154897f7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:32.979272 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1155/2287] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:32.990532 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264123be-9080-4597-a0c1-b1d5ef46c1d4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:32.992072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-264123be-9080-4597-a0c1-b1d5ef46c1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:33.046637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1b5fe72d-e2e4-4b07-8fe4-916807611cb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4584d5a6ba0c431cb7daad16154897f7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:33.047687 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1b5fe72d-e2e4-4b07-8fe4-916807611cb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4584d5a6ba0c431cb7daad16154897f7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:33.058385 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1133/2288] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:08:32 2022] DELETE /identity/v3/users/4584d5a6ba0c431cb7daad16154897f7 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:08:33.075311 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264123be-9080-4597-a0c1-b1d5ef46c1d4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:33.146135 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91eeee3-6e87-4691-9699-c49ca5dbd0d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:33.146962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91eeee3-6e87-4691-9699-c49ca5dbd0d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:33.146962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91eeee3-6e87-4691-9699-c49ca5dbd0d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:33.146962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91eeee3-6e87-4691-9699-c49ca5dbd0d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:33.148663 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d91eeee3-6e87-4691-9699-c49ca5dbd0d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:33.300847 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91eeee3-6e87-4691-9699-c49ca5dbd0d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:33.302800 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91eeee3-6e87-4691-9699-c49ca5dbd0d9 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:33.328281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1156/2289] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:08:33.583416 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:33.864454 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5fe72d-e2e4-4b07-8fe4-916807611cb9 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:33.866182 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1b5fe72d-e2e4-4b07-8fe4-916807611cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:33.879681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5fe72d-e2e4-4b07-8fe4-916807611cb9 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:33.886224 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fb788-1a5f-4a73-a598-d801dc1df72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:33.886873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fb788-1a5f-4a73-a598-d801dc1df72a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:33.890715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fb788-1a5f-4a73-a598-d801dc1df72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:33.890715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fb788-1a5f-4a73-a598-d801dc1df72a admin admin] PATH_INFO: `/v3/projects/8c14b4b05b8d44f8837bf964784ec2a2` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:33.890715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7fb788-1a5f-4a73-a598-d801dc1df72a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c14b4b05b8d44f8837bf964784ec2a2)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:33.891414 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7fb788-1a5f-4a73-a598-d801dc1df72a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:34.069459 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4f7fb788-1a5f-4a73-a598-d801dc1df72a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c14b4b05b8d44f8837bf964784ec2a2', 'request_id': 'req-4f7fb788-1a5f-4a73-a598-d801dc1df72a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:34.348639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4f7fb788-1a5f-4a73-a598-d801dc1df72a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c14b4b05b8d44f8837bf964784ec2a2', 'request_id': 'req-4f7fb788-1a5f-4a73-a598-d801dc1df72a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:34.357251 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4f7fb788-1a5f-4a73-a598-d801dc1df72a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c14b4b05b8d44f8837bf964784ec2a2', 'request_id': 'req-4f7fb788-1a5f-4a73-a598-d801dc1df72a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:34.371304 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4f7fb788-1a5f-4a73-a598-d801dc1df72a admin admin] The token cache is being invalidate because project 8c14b4b05b8d44f8837bf964784ec2a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:08:34.371617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4f7fb788-1a5f-4a73-a598-d801dc1df72a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:34.372446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4f7fb788-1a5f-4a73-a598-d801dc1df72a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:08:34.382189 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af87c8d-4714-414e-b6e8-bd9744284504 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:34.382678 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af87c8d-4714-414e-b6e8-bd9744284504 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:34.382758 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af87c8d-4714-414e-b6e8-bd9744284504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:34.387085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1134/2290] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:08:33 2022] DELETE /identity/v3/projects/8c14b4b05b8d44f8837bf964784ec2a2 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:34.415926 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8af87c8d-4714-414e-b6e8-bd9744284504 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:34.418848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8af87c8d-4714-414e-b6e8-bd9744284504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:34.527229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1157/2291] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:34.537102 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85128b-164f-4fce-9774-99cba1c1a04e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:34.537565 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85128b-164f-4fce-9774-99cba1c1a04e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:34.539035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85128b-164f-4fce-9774-99cba1c1a04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:34.565427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9f85128b-164f-4fce-9774-99cba1c1a04e None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:34.567058 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9f85128b-164f-4fce-9774-99cba1c1a04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:34.577577 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1135/2292] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:08:34.581832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af87c8d-4714-414e-b6e8-bd9744284504 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:34.593718 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af87c8d-4714-414e-b6e8-bd9744284504 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:34.600429 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e644672-1668-4cf4-9bdf-025a08e2100d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:34.601113 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e644672-1668-4cf4-9bdf-025a08e2100d admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:34.601392 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e644672-1668-4cf4-9bdf-025a08e2100d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:34.601661 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e644672-1668-4cf4-9bdf-025a08e2100d admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:34.602348 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e644672-1668-4cf4-9bdf-025a08e2100d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:08:34.603606 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e644672-1668-4cf4-9bdf-025a08e2100d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:34.606300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e644672-1668-4cf4-9bdf-025a08e2100d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:34.634187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1158/2293] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 12:08:34.635292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85128b-164f-4fce-9774-99cba1c1a04e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:34.648347 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85128b-164f-4fce-9774-99cba1c1a04e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:34.656318 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed43d5f-e665-46fb-896c-abfe77bae28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:34.657120 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed43d5f-e665-46fb-896c-abfe77bae28a admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:34.658167 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed43d5f-e665-46fb-896c-abfe77bae28a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:34.658461 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed43d5f-e665-46fb-896c-abfe77bae28a admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:34.659912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed43d5f-e665-46fb-896c-abfe77bae28a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:34.662646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed43d5f-e665-46fb-896c-abfe77bae28a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:35.150752 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:35.160214 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1136/2294] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:08:34 2022] POST /identity/v3/projects => generated 377 bytes in 526 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:08:35.166053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e644672-1668-4cf4-9bdf-025a08e2100d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:35.185643 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e644672-1668-4cf4-9bdf-025a08e2100d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:35.226009 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:35.272665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b0ce4-43e4-4b9a-a52d-91013016be43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:35.273286 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b0ce4-43e4-4b9a-a52d-91013016be43 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:35.273610 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b0ce4-43e4-4b9a-a52d-91013016be43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:35.274217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b0ce4-43e4-4b9a-a52d-91013016be43 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:35.274771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b0ce4-43e4-4b9a-a52d-91013016be43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:35.276746 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b0ce4-43e4-4b9a-a52d-91013016be43 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:35.398313 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1159/2295] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:08:35 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:08:35.402785 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed43d5f-e665-46fb-896c-abfe77bae28a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:35.414045 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed43d5f-e665-46fb-896c-abfe77bae28a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:35.419206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10707e99-848b-4a9e-ae97-b7bcd78b4bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:35.419823 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10707e99-848b-4a9e-ae97-b7bcd78b4bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:35.420139 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10707e99-848b-4a9e-ae97-b7bcd78b4bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:35.420416 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10707e99-848b-4a9e-ae97-b7bcd78b4bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:35.421070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10707e99-848b-4a9e-ae97-b7bcd78b4bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:35.422879 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10707e99-848b-4a9e-ae97-b7bcd78b4bd3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:35.434831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1137/2296] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:35.441242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b0ce4-43e4-4b9a-a52d-91013016be43 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:35.451660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b0ce4-43e4-4b9a-a52d-91013016be43 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:35.457555 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0049b1-4008-4707-9fb0-58546ce90969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:35.458300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0049b1-4008-4707-9fb0-58546ce90969 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:35.458641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0049b1-4008-4707-9fb0-58546ce90969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:35.458918 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0049b1-4008-4707-9fb0-58546ce90969 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:35.459589 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0049b1-4008-4707-9fb0-58546ce90969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:35.461286 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0049b1-4008-4707-9fb0-58546ce90969 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:35.472788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1160/2297] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:35.480976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10707e99-848b-4a9e-ae97-b7bcd78b4bd3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:35.492479 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10707e99-848b-4a9e-ae97-b7bcd78b4bd3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:35.498089 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c9cbc-7b6b-4930-91be-0a858b7c4ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:35.498801 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c9cbc-7b6b-4930-91be-0a858b7c4ff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:35.499087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c9cbc-7b6b-4930-91be-0a858b7c4ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:35.499368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c9cbc-7b6b-4930-91be-0a858b7c4ff0 admin admin] PATH_INFO: `/v3/projects/cc74efd9f7be4e65bd37bb9f2b888bab/users/8a662352635643058a12281e9c33a03d/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:35.522802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c9cbc-7b6b-4930-91be-0a858b7c4ff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc74efd9f7be4e65bd37bb9f2b888bab, user_id=8a662352635643058a12281e9c33a03d, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:35.526294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c9cbc-7b6b-4930-91be-0a858b7c4ff0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:35.653088 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad2299-f429-495d-8ae7-4537f4a45db5 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:35.653265 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad2299-f429-495d-8ae7-4537f4a45db5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:35.653544 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad2299-f429-495d-8ae7-4537f4a45db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:35.665338 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1138/2298] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:08:35 2022] PUT /identity/v3/projects/cc74efd9f7be4e65bd37bb9f2b888bab/users/8a662352635643058a12281e9c33a03d/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:08:35.676513 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4291fc0-d073-44a1-ac4f-385c3005cfa4 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:35.676830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4291fc0-d073-44a1-ac4f-385c3005cfa4 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:35.677107 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4291fc0-d073-44a1-ac4f-385c3005cfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:35.721781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-20ad2299-f429-495d-8ae7-4537f4a45db5 None None] MFA Rules not processed for user `8fe575351bdc456682db592f0c723ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:35.723306 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-20ad2299-f429-495d-8ae7-4537f4a45db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:35.769505 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d4291fc0-d073-44a1-ac4f-385c3005cfa4 None None] MFA Rules not processed for user `8a662352635643058a12281e9c33a03d`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:35.771329 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d4291fc0-d073-44a1-ac4f-385c3005cfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:35.817823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1161/2299] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:35.831052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad2299-f429-495d-8ae7-4537f4a45db5 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:35.832769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-20ad2299-f429-495d-8ae7-4537f4a45db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:35.905626 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1139/2300] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:08:35.911276 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4291fc0-d073-44a1-ac4f-385c3005cfa4 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:35.940134 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad2299-f429-495d-8ae7-4537f4a45db5 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:35.945366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34fdf3-398d-4b75-8b4a-ca6237ddca9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:35.946398 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34fdf3-398d-4b75-8b4a-ca6237ddca9a service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:35.946702 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34fdf3-398d-4b75-8b4a-ca6237ddca9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:35.947063 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34fdf3-398d-4b75-8b4a-ca6237ddca9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:35.960052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34fdf3-398d-4b75-8b4a-ca6237ddca9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:35.962303 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34fdf3-398d-4b75-8b4a-ca6237ddca9a service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:35.973151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1162/2301] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:08:36.011081 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4291fc0-d073-44a1-ac4f-385c3005cfa4 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:36.016592 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34acfeef-e61d-48f0-a196-6002f32be9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:36.017935 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acfeef-e61d-48f0-a196-6002f32be9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:36.017935 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acfeef-e61d-48f0-a196-6002f32be9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:36.017935 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acfeef-e61d-48f0-a196-6002f32be9b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:36.018541 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34acfeef-e61d-48f0-a196-6002f32be9b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:36.020556 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34acfeef-e61d-48f0-a196-6002f32be9b2 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:36.209641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1140/2302] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:08:35 2022] POST /identity/v3/projects => generated 377 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:08:36.214185 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34fdf3-398d-4b75-8b4a-ca6237ddca9a service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:36.225914 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34fdf3-398d-4b75-8b4a-ca6237ddca9a service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:36.268872 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:36.289604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ece1ff-595f-44b3-ba5d-3bae085b31af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:36.291084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ece1ff-595f-44b3-ba5d-3bae085b31af admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:36.291084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ece1ff-595f-44b3-ba5d-3bae085b31af admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:36.291084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ece1ff-595f-44b3-ba5d-3bae085b31af admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:36.291997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ece1ff-595f-44b3-ba5d-3bae085b31af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:36.293760 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ece1ff-595f-44b3-ba5d-3bae085b31af admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:36.400234 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1163/2303] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:08:36 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:08:36.403609 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34acfeef-e61d-48f0-a196-6002f32be9b2 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:36.417739 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34acfeef-e61d-48f0-a196-6002f32be9b2 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:36.425098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcaa3f-73d7-4739-b8b9-f28348e50ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:36.425741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcaa3f-73d7-4739-b8b9-f28348e50ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:36.426064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcaa3f-73d7-4739-b8b9-f28348e50ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:36.426345 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcaa3f-73d7-4739-b8b9-f28348e50ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:36.427041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fcaa3f-73d7-4739-b8b9-f28348e50ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:36.429571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fcaa3f-73d7-4739-b8b9-f28348e50ea9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:36.442960 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1141/2304] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:36.446998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ece1ff-595f-44b3-ba5d-3bae085b31af admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:36.458723 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ece1ff-595f-44b3-ba5d-3bae085b31af admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:36.464054 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7936e71-3f49-4ab5-9dbf-e3736a262c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:36.464596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7936e71-3f49-4ab5-9dbf-e3736a262c12 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:36.464884 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7936e71-3f49-4ab5-9dbf-e3736a262c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:36.465163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7936e71-3f49-4ab5-9dbf-e3736a262c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:36.465823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7936e71-3f49-4ab5-9dbf-e3736a262c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:36.467956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7936e71-3f49-4ab5-9dbf-e3736a262c12 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:36.485258 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1164/2305] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:08:36.488071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcaa3f-73d7-4739-b8b9-f28348e50ea9 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:36.500907 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcaa3f-73d7-4739-b8b9-f28348e50ea9 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:36.507691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ed536-bcec-41c7-8f92-bb83fbf6d57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:36.508418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ed536-bcec-41c7-8f92-bb83fbf6d57b admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:36.508709 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ed536-bcec-41c7-8f92-bb83fbf6d57b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:36.509014 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ed536-bcec-41c7-8f92-bb83fbf6d57b admin admin] PATH_INFO: `/v3/projects/cfc2c634bda643ff95207a22038e624d/users/1e4e1f357c034919b4a4f7f0ccce799e/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:36.530548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ed536-bcec-41c7-8f92-bb83fbf6d57b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfc2c634bda643ff95207a22038e624d, user_id=1e4e1f357c034919b4a4f7f0ccce799e, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:36.533582 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ed536-bcec-41c7-8f92-bb83fbf6d57b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:36.640634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1142/2306] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:08:36 2022] PUT /identity/v3/projects/cfc2c634bda643ff95207a22038e624d/users/1e4e1f357c034919b4a4f7f0ccce799e/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:08:36.652710 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f49b0-9f27-4a71-9323-9755074bd351 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:36.653075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f49b0-9f27-4a71-9323-9755074bd351 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:36.653415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f49b0-9f27-4a71-9323-9755074bd351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:36.713503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a02f49b0-9f27-4a71-9323-9755074bd351 None None] MFA Rules not processed for user `1e4e1f357c034919b4a4f7f0ccce799e`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:36.723701 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a02f49b0-9f27-4a71-9323-9755074bd351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:36.919118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ed536-bcec-41c7-8f92-bb83fbf6d57b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:36.920608 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1165/2307] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:36.930775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:37.033317 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ed536-bcec-41c7-8f92-bb83fbf6d57b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:37.038721 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441d8a4-1fda-4e5d-9f59-6aa34ba8b727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:37.039834 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441d8a4-1fda-4e5d-9f59-6aa34ba8b727 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:37.039834 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441d8a4-1fda-4e5d-9f59-6aa34ba8b727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:37.040504 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441d8a4-1fda-4e5d-9f59-6aa34ba8b727 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:37.043142 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441d8a4-1fda-4e5d-9f59-6aa34ba8b727 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:37.047504 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441d8a4-1fda-4e5d-9f59-6aa34ba8b727 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:37.209641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1143/2308] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:08:36 2022] POST /identity/v3/projects => generated 377 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:08:37.210897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f49b0-9f27-4a71-9323-9755074bd351 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:37.224498 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f49b0-9f27-4a71-9323-9755074bd351 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:37.307402 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3daa99-731a-46ed-8671-61332ba356c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:37.308027 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3daa99-731a-46ed-8671-61332ba356c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:37.310148 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3daa99-731a-46ed-8671-61332ba356c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:37.310451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3daa99-731a-46ed-8671-61332ba356c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:37.326218 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3daa99-731a-46ed-8671-61332ba356c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:37.326218 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3daa99-731a-46ed-8671-61332ba356c5 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:37.465423 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1166/2309] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:08:37 2022] POST /identity/v3/users => generated 295 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:08:37.471749 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441d8a4-1fda-4e5d-9f59-6aa34ba8b727 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:37.485635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441d8a4-1fda-4e5d-9f59-6aa34ba8b727 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:37.490914 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190d8ed-5526-4b23-8bb0-1ecd60d2f0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:37.491544 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190d8ed-5526-4b23-8bb0-1ecd60d2f0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:37.491832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190d8ed-5526-4b23-8bb0-1ecd60d2f0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:37.492114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190d8ed-5526-4b23-8bb0-1ecd60d2f0b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:37.492772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190d8ed-5526-4b23-8bb0-1ecd60d2f0b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:37.494557 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190d8ed-5526-4b23-8bb0-1ecd60d2f0b2 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:37.506479 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1144/2310] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:37.510832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3daa99-731a-46ed-8671-61332ba356c5 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:37.522949 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3daa99-731a-46ed-8671-61332ba356c5 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:37.527859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cd529-4bcd-44ff-bda2-2c02df3adfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:37.528520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cd529-4bcd-44ff-bda2-2c02df3adfb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:37.528807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cd529-4bcd-44ff-bda2-2c02df3adfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:37.529121 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cd529-4bcd-44ff-bda2-2c02df3adfb0 admin admin] PATH_INFO: `/v3/projects/f7b6fbeb60db4ef99d4dbfed1291def4/users/b6179196009a4bde8bf80e3b7bb01360/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:37.549813 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cd529-4bcd-44ff-bda2-2c02df3adfb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b6fbeb60db4ef99d4dbfed1291def4, user_id=b6179196009a4bde8bf80e3b7bb01360, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:37.552668 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cd529-4bcd-44ff-bda2-2c02df3adfb0 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:37.661373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1167/2311] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:08:37 2022] PUT /identity/v3/projects/f7b6fbeb60db4ef99d4dbfed1291def4/users/b6179196009a4bde8bf80e3b7bb01360/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:37.666637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190d8ed-5526-4b23-8bb0-1ecd60d2f0b2 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:37.678407 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190d8ed-5526-4b23-8bb0-1ecd60d2f0b2 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:37.744036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416128fb-06db-4541-8ffa-6a320e7560fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:37.744655 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416128fb-06db-4541-8ffa-6a320e7560fa admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:37.744941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416128fb-06db-4541-8ffa-6a320e7560fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:37.745224 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416128fb-06db-4541-8ffa-6a320e7560fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:37.745948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416128fb-06db-4541-8ffa-6a320e7560fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:37.747570 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416128fb-06db-4541-8ffa-6a320e7560fa admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:37.760631 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1145/2312] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:08:37.765495 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cd529-4bcd-44ff-bda2-2c02df3adfb0 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:37.779821 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cd529-4bcd-44ff-bda2-2c02df3adfb0 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:37.784608 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c07f22-4725-40cc-91db-1d2b16178d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:37.785285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c07f22-4725-40cc-91db-1d2b16178d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:37.785546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c07f22-4725-40cc-91db-1d2b16178d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:37.785819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c07f22-4725-40cc-91db-1d2b16178d7a admin admin] PATH_INFO: `/v3/projects/f7b6fbeb60db4ef99d4dbfed1291def4/users/b6179196009a4bde8bf80e3b7bb01360/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:37.787908 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c07f22-4725-40cc-91db-1d2b16178d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b6fbeb60db4ef99d4dbfed1291def4, user_id=b6179196009a4bde8bf80e3b7bb01360, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:37.790647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c07f22-4725-40cc-91db-1d2b16178d7a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:37.797747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:37.802504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1168/2313] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:08:37 2022] PUT /identity/v3/projects/f7b6fbeb60db4ef99d4dbfed1291def4/users/b6179196009a4bde8bf80e3b7bb01360/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:08:37.813259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded73e0-d384-40d4-b89a-a88e52ac0da8 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:37.813464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded73e0-d384-40d4-b89a-a88e52ac0da8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:37.814280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded73e0-d384-40d4-b89a-a88e52ac0da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:37.847427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-2ded73e0-d384-40d4-b89a-a88e52ac0da8 None None] MFA Rules not processed for user `b6179196009a4bde8bf80e3b7bb01360`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:37.848759 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2ded73e0-d384-40d4-b89a-a88e52ac0da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:37.925415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1146/2314] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:37.944960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c07f22-4725-40cc-91db-1d2b16178d7a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:38.024435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c07f22-4725-40cc-91db-1d2b16178d7a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:38.028994 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae3efc-375d-4be3-9d0b-c00b85a8a1fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:38.029636 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae3efc-375d-4be3-9d0b-c00b85a8a1fd service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:38.029957 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae3efc-375d-4be3-9d0b-c00b85a8a1fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:38.030739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae3efc-375d-4be3-9d0b-c00b85a8a1fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:38.102039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae3efc-375d-4be3-9d0b-c00b85a8a1fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:38.103553 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae3efc-375d-4be3-9d0b-c00b85a8a1fd service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:38.116470 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:38.116712 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1169/2315] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:08:40.079492 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2d2a4-f3bf-4d0a-8430-e9b156a09861 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:40.080101 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2d2a4-f3bf-4d0a-8430-e9b156a09861 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:40.080101 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2d2a4-f3bf-4d0a-8430-e9b156a09861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:40.103224 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e9d2d2a4-f3bf-4d0a-8430-e9b156a09861 None None] MFA Rules not processed for user `8fe575351bdc456682db592f0c723ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:08:40.107125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e9d2d2a4-f3bf-4d0a-8430-e9b156a09861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:40.173560 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1147/2316] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:08:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:08:42.882445 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae3efc-375d-4be3-9d0b-c00b85a8a1fd service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:42.884337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-84ae3efc-375d-4be3-9d0b-c00b85a8a1fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:08:42.897758 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae3efc-375d-4be3-9d0b-c00b85a8a1fd service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:42.902380 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10473f1-9eee-479f-bc04-d7ccf4da0ac5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:42.903022 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10473f1-9eee-479f-bc04-d7ccf4da0ac5 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:42.903304 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10473f1-9eee-479f-bc04-d7ccf4da0ac5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:42.903628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10473f1-9eee-479f-bc04-d7ccf4da0ac5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:42.981877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10473f1-9eee-479f-bc04-d7ccf4da0ac5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:42.983757 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10473f1-9eee-479f-bc04-d7ccf4da0ac5 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:42.994973 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1170/2317] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:08:45.289642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2d2a4-f3bf-4d0a-8430-e9b156a09861 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:45.300370 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2d2a4-f3bf-4d0a-8430-e9b156a09861 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:45.304124 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26f960-4e34-4feb-a827-c16ba69782d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:45.304739 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26f960-4e34-4feb-a827-c16ba69782d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:45.305026 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26f960-4e34-4feb-a827-c16ba69782d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:45.305300 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26f960-4e34-4feb-a827-c16ba69782d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:45.315229 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26f960-4e34-4feb-a827-c16ba69782d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:45.316698 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26f960-4e34-4feb-a827-c16ba69782d3 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:08:45.326877 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1148/2318] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:08:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:08:52.990545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=76 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:55.322239 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=32 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:08:59.871920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10473f1-9eee-479f-bc04-d7ccf4da0ac5 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:08:59.973140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10473f1-9eee-479f-bc04-d7ccf4da0ac5 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:08:59.982649 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15730458-2b48-403f-8bb8-30274fac908a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:08:59.983319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15730458-2b48-403f-8bb8-30274fac908a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:08:59.987635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15730458-2b48-403f-8bb8-30274fac908a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:08:59.987635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15730458-2b48-403f-8bb8-30274fac908a admin admin] PATH_INFO: `/v3/users/8a662352635643058a12281e9c33a03d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:08:59.987635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15730458-2b48-403f-8bb8-30274fac908a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a662352635643058a12281e9c33a03d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:08:59.989211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15730458-2b48-403f-8bb8-30274fac908a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:09:00.201156 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-15730458-2b48-403f-8bb8-30274fac908a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a662352635643058a12281e9c33a03d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:00.214741 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-15730458-2b48-403f-8bb8-30274fac908a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a662352635643058a12281e9c33a03d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:00.224597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-15730458-2b48-403f-8bb8-30274fac908a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a662352635643058a12281e9c33a03d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:00.292807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-15730458-2b48-403f-8bb8-30274fac908a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a662352635643058a12281e9c33a03d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:00.293608 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-15730458-2b48-403f-8bb8-30274fac908a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a662352635643058a12281e9c33a03d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:00.302576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1171/2319] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:08:59 2022] DELETE /identity/v3/users/8a662352635643058a12281e9c33a03d => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:09:00.308144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8544bc-8e46-4e99-a13c-05cb59282bff None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:00.308876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8544bc-8e46-4e99-a13c-05cb59282bff None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:00.309387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8544bc-8e46-4e99-a13c-05cb59282bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:00.332901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-0e8544bc-8e46-4e99-a13c-05cb59282bff None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:09:00.334178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0e8544bc-8e46-4e99-a13c-05cb59282bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:09:00.417917 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1149/2320] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:09:00.431693 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15730458-2b48-403f-8bb8-30274fac908a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:09:00.434193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-15730458-2b48-403f-8bb8-30274fac908a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:09:00.527057 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15730458-2b48-403f-8bb8-30274fac908a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:09:00.545098 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94def95c-56d9-44d5-941c-e54cd8bf4ea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:09:00.546288 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94def95c-56d9-44d5-941c-e54cd8bf4ea1 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:00.546652 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94def95c-56d9-44d5-941c-e54cd8bf4ea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:00.547811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94def95c-56d9-44d5-941c-e54cd8bf4ea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:00.579199 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94def95c-56d9-44d5-941c-e54cd8bf4ea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:09:00.580910 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94def95c-56d9-44d5-941c-e54cd8bf4ea1 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:09:00.614371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1172/2321] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:09:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:09:01.194492 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8544bc-8e46-4e99-a13c-05cb59282bff None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:09:01.197418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0e8544bc-8e46-4e99-a13c-05cb59282bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:09:01.236792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8544bc-8e46-4e99-a13c-05cb59282bff None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:09:01.247008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c85f11-9bfa-461d-a96c-dcfd0ed9a16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:09:01.247984 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c85f11-9bfa-461d-a96c-dcfd0ed9a16f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:01.248358 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c85f11-9bfa-461d-a96c-dcfd0ed9a16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:01.248776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c85f11-9bfa-461d-a96c-dcfd0ed9a16f admin admin] PATH_INFO: `/v3/projects/cc74efd9f7be4e65bd37bb9f2b888bab` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:01.250583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c85f11-9bfa-461d-a96c-dcfd0ed9a16f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc74efd9f7be4e65bd37bb9f2b888bab)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:09:01.253844 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c85f11-9bfa-461d-a96c-dcfd0ed9a16f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:09:01.397203 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-94c85f11-9bfa-461d-a96c-dcfd0ed9a16f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc74efd9f7be4e65bd37bb9f2b888bab', 'request_id': 'req-94c85f11-9bfa-461d-a96c-dcfd0ed9a16f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:01.470733 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-94c85f11-9bfa-461d-a96c-dcfd0ed9a16f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc74efd9f7be4e65bd37bb9f2b888bab', 'request_id': 'req-94c85f11-9bfa-461d-a96c-dcfd0ed9a16f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:01.478179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-94c85f11-9bfa-461d-a96c-dcfd0ed9a16f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc74efd9f7be4e65bd37bb9f2b888bab', 'request_id': 'req-94c85f11-9bfa-461d-a96c-dcfd0ed9a16f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:01.492815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-94c85f11-9bfa-461d-a96c-dcfd0ed9a16f admin admin] The token cache is being invalidate because project cc74efd9f7be4e65bd37bb9f2b888bab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:09:01.493012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-94c85f11-9bfa-461d-a96c-dcfd0ed9a16f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:01.494060 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-94c85f11-9bfa-461d-a96c-dcfd0ed9a16f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:01.495702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1150/2322] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:09:01 2022] DELETE /identity/v3/projects/cc74efd9f7be4e65bd37bb9f2b888bab => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:09:01.499915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94def95c-56d9-44d5-941c-e54cd8bf4ea1 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:09:01.514195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94def95c-56d9-44d5-941c-e54cd8bf4ea1 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:09:01.546322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:09:01.607533 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7642dff-7f30-46f6-baa4-f51186b5184b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:09:01.608672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7642dff-7f30-46f6-baa4-f51186b5184b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:01.609188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7642dff-7f30-46f6-baa4-f51186b5184b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:01.609299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7642dff-7f30-46f6-baa4-f51186b5184b admin admin] PATH_INFO: `/v3/users/1e4e1f357c034919b4a4f7f0ccce799e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:01.610569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7642dff-7f30-46f6-baa4-f51186b5184b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e4e1f357c034919b4a4f7f0ccce799e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:09:01.612911 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7642dff-7f30-46f6-baa4-f51186b5184b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:09:01.859440 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f7642dff-7f30-46f6-baa4-f51186b5184b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e4e1f357c034919b4a4f7f0ccce799e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:01.879113 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f7642dff-7f30-46f6-baa4-f51186b5184b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e4e1f357c034919b4a4f7f0ccce799e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:01.903066 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f7642dff-7f30-46f6-baa4-f51186b5184b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e4e1f357c034919b4a4f7f0ccce799e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:02.030589 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f7642dff-7f30-46f6-baa4-f51186b5184b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e4e1f357c034919b4a4f7f0ccce799e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:02.031771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f7642dff-7f30-46f6-baa4-f51186b5184b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e4e1f357c034919b4a4f7f0ccce799e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:02.039843 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1173/2323] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:09:01 2022] DELETE /identity/v3/users/1e4e1f357c034919b4a4f7f0ccce799e => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:09:02.809472 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c85f11-9bfa-461d-a96c-dcfd0ed9a16f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:09:02.811807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-94c85f11-9bfa-461d-a96c-dcfd0ed9a16f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:09:02.828899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c85f11-9bfa-461d-a96c-dcfd0ed9a16f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:09:02.929996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabf556-9eec-4ab5-a458-9bbc2c997d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:09:02.931320 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabf556-9eec-4ab5-a458-9bbc2c997d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:02.931925 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabf556-9eec-4ab5-a458-9bbc2c997d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:02.932422 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabf556-9eec-4ab5-a458-9bbc2c997d11 admin admin] PATH_INFO: `/v3/projects/cfc2c634bda643ff95207a22038e624d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:02.935044 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabf556-9eec-4ab5-a458-9bbc2c997d11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfc2c634bda643ff95207a22038e624d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:09:02.937397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabf556-9eec-4ab5-a458-9bbc2c997d11 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:09:03.077232 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:09:03.101815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4fabf556-9eec-4ab5-a458-9bbc2c997d11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfc2c634bda643ff95207a22038e624d', 'request_id': 'req-4fabf556-9eec-4ab5-a458-9bbc2c997d11'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:03.188829 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4fabf556-9eec-4ab5-a458-9bbc2c997d11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfc2c634bda643ff95207a22038e624d', 'request_id': 'req-4fabf556-9eec-4ab5-a458-9bbc2c997d11'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:03.205488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4fabf556-9eec-4ab5-a458-9bbc2c997d11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfc2c634bda643ff95207a22038e624d', 'request_id': 'req-4fabf556-9eec-4ab5-a458-9bbc2c997d11'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:03.220270 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4fabf556-9eec-4ab5-a458-9bbc2c997d11 admin admin] The token cache is being invalidate because project cfc2c634bda643ff95207a22038e624d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:09:03.220656 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4fabf556-9eec-4ab5-a458-9bbc2c997d11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:03.229142 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4fabf556-9eec-4ab5-a458-9bbc2c997d11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:03.229142 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1151/2324] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:09:02 2022] DELETE /identity/v3/projects/cfc2c634bda643ff95207a22038e624d => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:09:03.232501 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7642dff-7f30-46f6-baa4-f51186b5184b admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:09:03.234057 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f7642dff-7f30-46f6-baa4-f51186b5184b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:09:03.249731 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7642dff-7f30-46f6-baa4-f51186b5184b admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:09:03.355790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b5ca1-ff87-4b90-9af7-f2112487af10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:09:03.356510 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b5ca1-ff87-4b90-9af7-f2112487af10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:03.356799 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b5ca1-ff87-4b90-9af7-f2112487af10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:03.357081 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b5ca1-ff87-4b90-9af7-f2112487af10 admin admin] PATH_INFO: `/v3/users/b6179196009a4bde8bf80e3b7bb01360` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:03.358339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b5ca1-ff87-4b90-9af7-f2112487af10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6179196009a4bde8bf80e3b7bb01360)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:09:03.360908 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b5ca1-ff87-4b90-9af7-f2112487af10 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:09:03.576042 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:09:03.578930 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9f4b5ca1-ff87-4b90-9af7-f2112487af10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6179196009a4bde8bf80e3b7bb01360'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:03.646764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9f4b5ca1-ff87-4b90-9af7-f2112487af10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6179196009a4bde8bf80e3b7bb01360'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:03.662095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9f4b5ca1-ff87-4b90-9af7-f2112487af10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6179196009a4bde8bf80e3b7bb01360'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:03.784092 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9f4b5ca1-ff87-4b90-9af7-f2112487af10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6179196009a4bde8bf80e3b7bb01360'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:03.785602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9f4b5ca1-ff87-4b90-9af7-f2112487af10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6179196009a4bde8bf80e3b7bb01360'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:03.802330 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1174/2325] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:09:03 2022] DELETE /identity/v3/users/b6179196009a4bde8bf80e3b7bb01360 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:09:04.412912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabf556-9eec-4ab5-a458-9bbc2c997d11 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:09:04.417263 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4fabf556-9eec-4ab5-a458-9bbc2c997d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:09:04.434779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabf556-9eec-4ab5-a458-9bbc2c997d11 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:09:04.530760 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e853d-23bb-4814-b5e9-8da8b190be4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:09:04.531467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e853d-23bb-4814-b5e9-8da8b190be4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:04.531760 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e853d-23bb-4814-b5e9-8da8b190be4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:04.532047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e853d-23bb-4814-b5e9-8da8b190be4c admin admin] PATH_INFO: `/v3/projects/f7b6fbeb60db4ef99d4dbfed1291def4` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:04.533301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e853d-23bb-4814-b5e9-8da8b190be4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7b6fbeb60db4ef99d4dbfed1291def4)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:09:04.535443 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e853d-23bb-4814-b5e9-8da8b190be4c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:09:04.738820 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a70e853d-23bb-4814-b5e9-8da8b190be4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7b6fbeb60db4ef99d4dbfed1291def4', 'request_id': 'req-a70e853d-23bb-4814-b5e9-8da8b190be4c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:04.847164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a70e853d-23bb-4814-b5e9-8da8b190be4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7b6fbeb60db4ef99d4dbfed1291def4', 'request_id': 'req-a70e853d-23bb-4814-b5e9-8da8b190be4c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:04.855205 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a70e853d-23bb-4814-b5e9-8da8b190be4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7b6fbeb60db4ef99d4dbfed1291def4', 'request_id': 'req-a70e853d-23bb-4814-b5e9-8da8b190be4c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:04.870204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a70e853d-23bb-4814-b5e9-8da8b190be4c admin admin] The token cache is being invalidate because project f7b6fbeb60db4ef99d4dbfed1291def4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:09:04.870503 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a70e853d-23bb-4814-b5e9-8da8b190be4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:04.871243 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a70e853d-23bb-4814-b5e9-8da8b190be4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:04.873312 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1152/2326] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:09:04 2022] DELETE /identity/v3/projects/f7b6fbeb60db4ef99d4dbfed1291def4 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:09:13.813062 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:DELETE=4,keystone:SELECT=13,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:09:14.866789 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:DELETE=13,keystone:SELECT=65,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:09:17.580913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b5ca1-ff87-4b90-9af7-f2112487af10 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:09:17.582609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9f4b5ca1-ff87-4b90-9af7-f2112487af10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:09:17.607322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b5ca1-ff87-4b90-9af7-f2112487af10 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:09:17.675560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ad921-9a57-46ca-a631-a55767d06de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:09:17.676204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ad921-9a57-46ca-a631-a55767d06de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:17.676463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ad921-9a57-46ca-a631-a55767d06de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:17.676739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ad921-9a57-46ca-a631-a55767d06de1 admin admin] PATH_INFO: `/v3/users/f41074ef6a544113b744dd90ed4e77bf` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:17.677869 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ad921-9a57-46ca-a631-a55767d06de1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f41074ef6a544113b744dd90ed4e77bf)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:09:17.679798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ad921-9a57-46ca-a631-a55767d06de1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:09:17.979033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f83ad921-9a57-46ca-a631-a55767d06de1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f41074ef6a544113b744dd90ed4e77bf'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:17.996680 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f83ad921-9a57-46ca-a631-a55767d06de1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f41074ef6a544113b744dd90ed4e77bf'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:18.007305 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f83ad921-9a57-46ca-a631-a55767d06de1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f41074ef6a544113b744dd90ed4e77bf'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:18.127698 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f83ad921-9a57-46ca-a631-a55767d06de1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f41074ef6a544113b744dd90ed4e77bf'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:18.128437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f83ad921-9a57-46ca-a631-a55767d06de1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f41074ef6a544113b744dd90ed4e77bf'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:18.135282 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1175/2327] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:09:17 2022] DELETE /identity/v3/users/f41074ef6a544113b744dd90ed4e77bf => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:09:18.140218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b21070-79da-4797-97f5-5cd8cc32c21b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:18.140703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b21070-79da-4797-97f5-5cd8cc32c21b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:18.140915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b21070-79da-4797-97f5-5cd8cc32c21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:18.160999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-01b21070-79da-4797-97f5-5cd8cc32c21b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:09:18.162288 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-01b21070-79da-4797-97f5-5cd8cc32c21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:09:18.202329 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ad921-9a57-46ca-a631-a55767d06de1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:09:18.203924 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f83ad921-9a57-46ca-a631-a55767d06de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:09:18.245218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1153/2328] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:09:18.258268 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b21070-79da-4797-97f5-5cd8cc32c21b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:09:18.260568 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-01b21070-79da-4797-97f5-5cd8cc32c21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:09:18.301512 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ad921-9a57-46ca-a631-a55767d06de1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:09:18.308649 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3674c4e-1729-448c-b821-33c2a3589103 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca10dd7ed714edb8b87017deecc41b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:09:18.309272 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3674c4e-1729-448c-b821-33c2a3589103 service placement] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:18.309551 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3674c4e-1729-448c-b821-33c2a3589103 service placement] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:18.309828 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3674c4e-1729-448c-b821-33c2a3589103 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:18.311719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b3674c4e-1729-448c-b821-33c2a3589103 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:09:18.353411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b21070-79da-4797-97f5-5cd8cc32c21b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:09:18.353679 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3674c4e-1729-448c-b821-33c2a3589103 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:09:18.353679 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3674c4e-1729-448c-b821-33c2a3589103 service placement] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:09:18.354023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c7180-b0f1-491e-af58-e7003ebb7a8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:09:18.354023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c7180-b0f1-491e-af58-e7003ebb7a8d service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:18.354023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c7180-b0f1-491e-af58-e7003ebb7a8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:18.354023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c7180-b0f1-491e-af58-e7003ebb7a8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:18.354582 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1176/2329] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:09:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:09:18.354731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c7180-b0f1-491e-af58-e7003ebb7a8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:09:18.354731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c7180-b0f1-491e-af58-e7003ebb7a8d service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:09:18.365800 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1154/2330] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:09:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:09:19.009766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3674c4e-1729-448c-b821-33c2a3589103 service placement] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:09:19.012483 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b3674c4e-1729-448c-b821-33c2a3589103 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:09:19.031663 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3674c4e-1729-448c-b821-33c2a3589103 service placement] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:09:19.036684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe9776-b372-45a3-9634-aa71d2b09696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:09:19.037352 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe9776-b372-45a3-9634-aa71d2b09696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:19.037672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe9776-b372-45a3-9634-aa71d2b09696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:19.038187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe9776-b372-45a3-9634-aa71d2b09696 admin admin] PATH_INFO: `/v3/projects/f6192727aecd471da46a4e01b07c3a39` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:19.039461 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe9776-b372-45a3-9634-aa71d2b09696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6192727aecd471da46a4e01b07c3a39)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:09:19.041755 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe9776-b372-45a3-9634-aa71d2b09696 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:09:19.053364 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:09:19.199203 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5ffe9776-b372-45a3-9634-aa71d2b09696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6192727aecd471da46a4e01b07c3a39', 'request_id': 'req-5ffe9776-b372-45a3-9634-aa71d2b09696'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:19.269058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5ffe9776-b372-45a3-9634-aa71d2b09696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6192727aecd471da46a4e01b07c3a39', 'request_id': 'req-5ffe9776-b372-45a3-9634-aa71d2b09696'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:19.277684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5ffe9776-b372-45a3-9634-aa71d2b09696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6192727aecd471da46a4e01b07c3a39', 'request_id': 'req-5ffe9776-b372-45a3-9634-aa71d2b09696'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:19.290002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5ffe9776-b372-45a3-9634-aa71d2b09696 admin admin] The token cache is being invalidate because project f6192727aecd471da46a4e01b07c3a39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:09:19.290450 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5ffe9776-b372-45a3-9634-aa71d2b09696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:19.291132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5ffe9776-b372-45a3-9634-aa71d2b09696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:09:19.292869 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1177/2331] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:09:19 2022] DELETE /identity/v3/projects/f6192727aecd471da46a4e01b07c3a39 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:09:19.298449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df21ee-18b0-4f9d-a6d2-e74bc6e44d20 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:19.298939 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df21ee-18b0-4f9d-a6d2-e74bc6e44d20 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:19.299034 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df21ee-18b0-4f9d-a6d2-e74bc6e44d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:19.326644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b7df21ee-18b0-4f9d-a6d2-e74bc6e44d20 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:09:19.327852 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b7df21ee-18b0-4f9d-a6d2-e74bc6e44d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:09:19.396053 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1155/2332] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:09:19.408093 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf924d-e3fa-4001-92a0-918b9974b582 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:19.408946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf924d-e3fa-4001-92a0-918b9974b582 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:19.409597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf924d-e3fa-4001-92a0-918b9974b582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:19.432517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-81cf924d-e3fa-4001-92a0-918b9974b582 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:09:19.433785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-81cf924d-e3fa-4001-92a0-918b9974b582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:09:19.442639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1178/2333] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:09:19.446997 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df21ee-18b0-4f9d-a6d2-e74bc6e44d20 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:09:19.458901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df21ee-18b0-4f9d-a6d2-e74bc6e44d20 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:09:19.463964 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13d656-c091-4f54-8e43-175ad7b5146e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:09:19.469523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13d656-c091-4f54-8e43-175ad7b5146e admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:19.469523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13d656-c091-4f54-8e43-175ad7b5146e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:19.469523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13d656-c091-4f54-8e43-175ad7b5146e admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:19.469523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13d656-c091-4f54-8e43-175ad7b5146e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:09:19.469523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13d656-c091-4f54-8e43-175ad7b5146e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:09:19.469523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13d656-c091-4f54-8e43-175ad7b5146e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:09:19.481230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1156/2334] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:09:19.486852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf924d-e3fa-4001-92a0-918b9974b582 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:09:19.498366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf924d-e3fa-4001-92a0-918b9974b582 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:09:19.504279 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bafda5-8648-4005-a179-963056f05aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:09:19.504903 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bafda5-8648-4005-a179-963056f05aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:19.505212 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bafda5-8648-4005-a179-963056f05aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:19.505542 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bafda5-8648-4005-a179-963056f05aa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:19.506239 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bafda5-8648-4005-a179-963056f05aa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:09:19.508173 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bafda5-8648-4005-a179-963056f05aa6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:09:19.666604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1179/2335] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:09:19 2022] POST /identity/v3/projects => generated 365 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:09:19.670656 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13d656-c091-4f54-8e43-175ad7b5146e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:09:19.681267 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13d656-c091-4f54-8e43-175ad7b5146e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:09:19.719748 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:09:19.748392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4fac56-6d73-401a-b184-253c1b481cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:09:19.749173 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fac56-6d73-401a-b184-253c1b481cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:19.749455 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fac56-6d73-401a-b184-253c1b481cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:19.753511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fac56-6d73-401a-b184-253c1b481cb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:19.753511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4fac56-6d73-401a-b184-253c1b481cb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:09:19.753511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4fac56-6d73-401a-b184-253c1b481cb6 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:09:19.835455 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1157/2336] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:09:19 2022] POST /identity/v3/users => generated 283 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:09:19.839782 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bafda5-8648-4005-a179-963056f05aa6 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:09:19.851517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bafda5-8648-4005-a179-963056f05aa6 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:09:19.856809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d8d38-4334-4892-bd67-f4c098b01b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:09:19.857424 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d8d38-4334-4892-bd67-f4c098b01b11 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:19.857704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d8d38-4334-4892-bd67-f4c098b01b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:19.858015 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d8d38-4334-4892-bd67-f4c098b01b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:19.858657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d8d38-4334-4892-bd67-f4c098b01b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:09:19.860891 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d8d38-4334-4892-bd67-f4c098b01b11 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:09:19.871760 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1180/2337] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:09:19.881767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4fac56-6d73-401a-b184-253c1b481cb6 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:09:19.892602 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4fac56-6d73-401a-b184-253c1b481cb6 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:09:19.898769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4da604-46b9-48c2-8964-fad2a3c250ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:09:19.899968 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4da604-46b9-48c2-8964-fad2a3c250ed admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:19.901033 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4da604-46b9-48c2-8964-fad2a3c250ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:19.901033 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4da604-46b9-48c2-8964-fad2a3c250ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:19.901230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4da604-46b9-48c2-8964-fad2a3c250ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:09:19.902822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4da604-46b9-48c2-8964-fad2a3c250ed admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:09:19.919283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1158/2338] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:09:19.920827 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d8d38-4334-4892-bd67-f4c098b01b11 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:09:19.932349 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d8d38-4334-4892-bd67-f4c098b01b11 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:09:19.937558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8867b97-de88-4de2-9c4a-8ff533301a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:09:19.938275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8867b97-de88-4de2-9c4a-8ff533301a94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:19.938557 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8867b97-de88-4de2-9c4a-8ff533301a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:19.938839 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8867b97-de88-4de2-9c4a-8ff533301a94 admin admin] PATH_INFO: `/v3/projects/129ac241dc4140a9865f951c0b7de064/users/b175841d29a546a7965faa045106ff9a/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:19.959507 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8867b97-de88-4de2-9c4a-8ff533301a94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=129ac241dc4140a9865f951c0b7de064, user_id=b175841d29a546a7965faa045106ff9a, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:09:19.963490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8867b97-de88-4de2-9c4a-8ff533301a94 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:09:20.001918 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78fbd7-05a9-4048-95ef-6fb96d8d866c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:20.001918 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78fbd7-05a9-4048-95ef-6fb96d8d866c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:20.001918 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78fbd7-05a9-4048-95ef-6fb96d8d866c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:20.022547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-6a78fbd7-05a9-4048-95ef-6fb96d8d866c None None] MFA Rules not processed for user `e133fb1749004c36a38ac028782294e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:09:20.023837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6a78fbd7-05a9-4048-95ef-6fb96d8d866c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:09:20.060411 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1181/2339] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:09:19 2022] PUT /identity/v3/projects/129ac241dc4140a9865f951c0b7de064/users/b175841d29a546a7965faa045106ff9a/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:09:20.082285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18a4d4-e9b0-4dca-941e-f349f4c928c0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:20.083230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18a4d4-e9b0-4dca-941e-f349f4c928c0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:20.086226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18a4d4-e9b0-4dca-941e-f349f4c928c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:20.092299 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1159/2340] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:09:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:09:20.122186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8b18a4d4-e9b0-4dca-941e-f349f4c928c0 None None] MFA Rules not processed for user `b175841d29a546a7965faa045106ff9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:09:20.123913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8b18a4d4-e9b0-4dca-941e-f349f4c928c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:09:20.199028 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1182/2341] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:09:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:09:20.211361 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78fbd7-05a9-4048-95ef-6fb96d8d866c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:09:20.212767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6a78fbd7-05a9-4048-95ef-6fb96d8d866c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:09:20.281549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78fbd7-05a9-4048-95ef-6fb96d8d866c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:09:20.369181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bc583-2a3b-486c-9add-3a528c79ea6c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b05d0e65e9645f3810a6d96c8fed9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:09:20.370840 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bc583-2a3b-486c-9add-3a528c79ea6c service glance] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:20.371413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bc583-2a3b-486c-9add-3a528c79ea6c service glance] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:20.371876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bc583-2a3b-486c-9add-3a528c79ea6c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:20.393007 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293bc583-2a3b-486c-9add-3a528c79ea6c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:09:20.394762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293bc583-2a3b-486c-9add-3a528c79ea6c service glance] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:09:20.417900 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1160/2342] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:09:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:09:21.149042 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759171aa-6252-46ee-a82f-fd9b61e1a374 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:09:21.150059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759171aa-6252-46ee-a82f-fd9b61e1a374 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:09:21.150332 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759171aa-6252-46ee-a82f-fd9b61e1a374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:09:21.173572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-759171aa-6252-46ee-a82f-fd9b61e1a374 None None] MFA Rules not processed for user `4bd4125ff6bb400bae1b94df7d2cfa17`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:09:21.174826 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-759171aa-6252-46ee-a82f-fd9b61e1a374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:09:21.191388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:09:21.256392 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1183/2343] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:09:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:09:30.402470 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:09:31.779059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=17 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:10:09.295345 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeaab5d-359c-4ec9-ae6d-8d2c5856ce6f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:10:09.295927 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeaab5d-359c-4ec9-ae6d-8d2c5856ce6f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:10:09.295927 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeaab5d-359c-4ec9-ae6d-8d2c5856ce6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:10:09.321532 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3eeaab5d-359c-4ec9-ae6d-8d2c5856ce6f None None] MFA Rules not processed for user `d76a164ed7e14d9eb2db016b68a84d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:10:09.322871 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3eeaab5d-359c-4ec9-ae6d-8d2c5856ce6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:10:09.390245 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1161/2344] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 14 12:10:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:10:09.441034 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759171aa-6252-46ee-a82f-fd9b61e1a374 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:10:09.442613 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-759171aa-6252-46ee-a82f-fd9b61e1a374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:10:09.521464 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759171aa-6252-46ee-a82f-fd9b61e1a374 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:10:09.526726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ec18f-430d-4dcf-b658-a05b14f59973 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1047c7d18c1466d989cd895117cd807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:10:09.527335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ec18f-430d-4dcf-b658-a05b14f59973 service swift] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:10:09.527614 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ec18f-430d-4dcf-b658-a05b14f59973 service swift] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:10:09.527895 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ec18f-430d-4dcf-b658-a05b14f59973 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:10:09.537452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ec18f-430d-4dcf-b658-a05b14f59973 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:10:09.539016 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ec18f-430d-4dcf-b658-a05b14f59973 service swift] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:10:09.547976 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1184/2345] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 14 12:10:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 12:10:19.381471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=25 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:10:19.544558 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=32 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:10:23.636745 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeaab5d-359c-4ec9-ae6d-8d2c5856ce6f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:10:23.715780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeaab5d-359c-4ec9-ae6d-8d2c5856ce6f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:10:23.721189 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4abb8-d171-4f3d-9e60-67ca4c79ed25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:10:23.722124 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4abb8-d171-4f3d-9e60-67ca4c79ed25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:10:23.722425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4abb8-d171-4f3d-9e60-67ca4c79ed25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:10:23.722705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4abb8-d171-4f3d-9e60-67ca4c79ed25 admin admin] PATH_INFO: `/v3/users/b175841d29a546a7965faa045106ff9a` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:10:23.724219 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4abb8-d171-4f3d-9e60-67ca4c79ed25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b175841d29a546a7965faa045106ff9a)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:10:23.725946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4abb8-d171-4f3d-9e60-67ca4c79ed25 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:10:23.999590 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-69d4abb8-d171-4f3d-9e60-67ca4c79ed25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b175841d29a546a7965faa045106ff9a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:24.015608 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-69d4abb8-d171-4f3d-9e60-67ca4c79ed25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b175841d29a546a7965faa045106ff9a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:24.025224 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-69d4abb8-d171-4f3d-9e60-67ca4c79ed25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b175841d29a546a7965faa045106ff9a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:24.127884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-69d4abb8-d171-4f3d-9e60-67ca4c79ed25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b175841d29a546a7965faa045106ff9a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:24.129108 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-69d4abb8-d171-4f3d-9e60-67ca4c79ed25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b175841d29a546a7965faa045106ff9a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:24.136804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1162/2346] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:10:23 2022] DELETE /identity/v3/users/b175841d29a546a7965faa045106ff9a => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:10:24.142243 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e7e25-c251-4404-97ee-151d3890e1be None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:10:24.142924 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e7e25-c251-4404-97ee-151d3890e1be None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:10:24.142924 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e7e25-c251-4404-97ee-151d3890e1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:10:24.163836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ec7e7e25-c251-4404-97ee-151d3890e1be None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:10:24.165214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ec7e7e25-c251-4404-97ee-151d3890e1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:10:24.234250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1185/2347] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:10:24.245426 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4abb8-d171-4f3d-9e60-67ca4c79ed25 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:10:24.246951 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-69d4abb8-d171-4f3d-9e60-67ca4c79ed25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:10:24.328429 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4abb8-d171-4f3d-9e60-67ca4c79ed25 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:10:24.334681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3007aa-7abc-47dd-a638-8ac92bea311e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:10:24.337931 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3007aa-7abc-47dd-a638-8ac92bea311e service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:10:24.341508 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3007aa-7abc-47dd-a638-8ac92bea311e service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:10:24.341508 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3007aa-7abc-47dd-a638-8ac92bea311e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:10:24.349628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3007aa-7abc-47dd-a638-8ac92bea311e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:10:24.352533 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3007aa-7abc-47dd-a638-8ac92bea311e service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:10:24.369700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1163/2348] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:10:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:10:24.941615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e7e25-c251-4404-97ee-151d3890e1be None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:10:24.952330 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ec7e7e25-c251-4404-97ee-151d3890e1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:10:24.955759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e7e25-c251-4404-97ee-151d3890e1be None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:10:24.960773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71304ee-a161-42fb-9956-80c80e226a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:10:24.961408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71304ee-a161-42fb-9956-80c80e226a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:10:24.961701 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71304ee-a161-42fb-9956-80c80e226a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:10:24.962019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71304ee-a161-42fb-9956-80c80e226a83 admin admin] PATH_INFO: `/v3/projects/129ac241dc4140a9865f951c0b7de064` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:10:24.963163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71304ee-a161-42fb-9956-80c80e226a83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=129ac241dc4140a9865f951c0b7de064)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:10:24.965718 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71304ee-a161-42fb-9956-80c80e226a83 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:10:25.134563 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b71304ee-a161-42fb-9956-80c80e226a83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '129ac241dc4140a9865f951c0b7de064', 'request_id': 'req-b71304ee-a161-42fb-9956-80c80e226a83'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:25.239676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b71304ee-a161-42fb-9956-80c80e226a83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '129ac241dc4140a9865f951c0b7de064', 'request_id': 'req-b71304ee-a161-42fb-9956-80c80e226a83'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:25.249008 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b71304ee-a161-42fb-9956-80c80e226a83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '129ac241dc4140a9865f951c0b7de064', 'request_id': 'req-b71304ee-a161-42fb-9956-80c80e226a83'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:25.265496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b71304ee-a161-42fb-9956-80c80e226a83 admin admin] The token cache is being invalidate because project 129ac241dc4140a9865f951c0b7de064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:10:25.266170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b71304ee-a161-42fb-9956-80c80e226a83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:25.267052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b71304ee-a161-42fb-9956-80c80e226a83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:25.269596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1186/2349] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:10:24 2022] DELETE /identity/v3/projects/129ac241dc4140a9865f951c0b7de064 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:10:34.362160 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:10:35.262173 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:10:35.611120 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce03b1e-b711-45bf-b2da-d7e9b48ad557 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:10:35.611687 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce03b1e-b711-45bf-b2da-d7e9b48ad557 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:10:35.611687 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce03b1e-b711-45bf-b2da-d7e9b48ad557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:10:35.631886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3ce03b1e-b711-45bf-b2da-d7e9b48ad557 None None] MFA Rules not processed for user `d76a164ed7e14d9eb2db016b68a84d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:10:35.633108 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3ce03b1e-b711-45bf-b2da-d7e9b48ad557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:10:35.691731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1164/2350] 10.222.0.62 () {58 vars in 967 bytes} [Wed Sep 14 12:10:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:10:35.704795 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71304ee-a161-42fb-9956-80c80e226a83 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:10:35.706260 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b71304ee-a161-42fb-9956-80c80e226a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:10:35.798314 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71304ee-a161-42fb-9956-80c80e226a83 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:10:35.803990 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f86ca1-8786-4054-8530-3d7a7649bd59 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1047c7d18c1466d989cd895117cd807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:10:35.804592 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f86ca1-8786-4054-8530-3d7a7649bd59 service swift] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:10:35.804871 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f86ca1-8786-4054-8530-3d7a7649bd59 service swift] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:10:35.805143 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f86ca1-8786-4054-8530-3d7a7649bd59 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:10:35.813246 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f86ca1-8786-4054-8530-3d7a7649bd59 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:10:35.814837 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f86ca1-8786-4054-8530-3d7a7649bd59 service swift] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:10:35.823269 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1187/2351] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 14 12:10:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 14 12:10:45.685081 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=25 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:10:45.819642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=32 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:10:47.583172 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce03b1e-b711-45bf-b2da-d7e9b48ad557 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:10:47.584683 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3ce03b1e-b711-45bf-b2da-d7e9b48ad557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:10:47.657149 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce03b1e-b711-45bf-b2da-d7e9b48ad557 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:10:47.661819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b24fa0-b533-498c-a49c-b433cb28c231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:10:47.662484 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b24fa0-b533-498c-a49c-b433cb28c231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:10:47.662770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b24fa0-b533-498c-a49c-b433cb28c231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:10:47.663078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b24fa0-b533-498c-a49c-b433cb28c231 admin admin] PATH_INFO: `/v3/users/e133fb1749004c36a38ac028782294e2` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:10:47.664985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b24fa0-b533-498c-a49c-b433cb28c231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e133fb1749004c36a38ac028782294e2)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:10:47.666972 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b24fa0-b533-498c-a49c-b433cb28c231 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:10:47.867513 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-54b24fa0-b533-498c-a49c-b433cb28c231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e133fb1749004c36a38ac028782294e2'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:47.880933 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-54b24fa0-b533-498c-a49c-b433cb28c231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e133fb1749004c36a38ac028782294e2'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:47.888244 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-54b24fa0-b533-498c-a49c-b433cb28c231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e133fb1749004c36a38ac028782294e2'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:47.994671 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-54b24fa0-b533-498c-a49c-b433cb28c231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e133fb1749004c36a38ac028782294e2'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:47.995314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-54b24fa0-b533-498c-a49c-b433cb28c231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e133fb1749004c36a38ac028782294e2'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:48.003226 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1165/2352] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:10:47 2022] DELETE /identity/v3/users/e133fb1749004c36a38ac028782294e2 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:10:48.275693 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f86ca1-8786-4054-8530-3d7a7649bd59 service swift] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:10:48.289150 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f86ca1-8786-4054-8530-3d7a7649bd59 service swift] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:10:48.348559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ec268-0626-4d2e-99dd-101472d7e090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:10:48.349508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ec268-0626-4d2e-99dd-101472d7e090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:10:48.349508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ec268-0626-4d2e-99dd-101472d7e090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:10:48.349751 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ec268-0626-4d2e-99dd-101472d7e090 admin admin] PATH_INFO: `/v3/projects/2e692d812c3842608e10769277212d85` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:10:48.350919 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ec268-0626-4d2e-99dd-101472d7e090 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e692d812c3842608e10769277212d85)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:10:48.352825 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ec268-0626-4d2e-99dd-101472d7e090 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:10:48.465004 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-522ec268-0626-4d2e-99dd-101472d7e090 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e692d812c3842608e10769277212d85', 'request_id': 'req-522ec268-0626-4d2e-99dd-101472d7e090'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:48.544923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-522ec268-0626-4d2e-99dd-101472d7e090 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e692d812c3842608e10769277212d85', 'request_id': 'req-522ec268-0626-4d2e-99dd-101472d7e090'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:48.552602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-522ec268-0626-4d2e-99dd-101472d7e090 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e692d812c3842608e10769277212d85', 'request_id': 'req-522ec268-0626-4d2e-99dd-101472d7e090'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:48.564393 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-522ec268-0626-4d2e-99dd-101472d7e090 admin admin] The token cache is being invalidate because project 2e692d812c3842608e10769277212d85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:10:48.564682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-522ec268-0626-4d2e-99dd-101472d7e090 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:48.565347 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-522ec268-0626-4d2e-99dd-101472d7e090 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:10:48.566953 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1188/2353] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:10:48 2022] DELETE /identity/v3/projects/2e692d812c3842608e10769277212d85 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:10:54.471480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197e055-bd2c-4eab-8e24-7dde40e8cd51 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:10:54.471987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197e055-bd2c-4eab-8e24-7dde40e8cd51 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:10:54.472159 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197e055-bd2c-4eab-8e24-7dde40e8cd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:10:54.492995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4197e055-bd2c-4eab-8e24-7dde40e8cd51 None None] MFA Rules not processed for user `d76a164ed7e14d9eb2db016b68a84d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:10:54.494373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4197e055-bd2c-4eab-8e24-7dde40e8cd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:10:54.546171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ec268-0626-4d2e-99dd-101472d7e090 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:10:54.547602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-522ec268-0626-4d2e-99dd-101472d7e090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:10:54.554446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1166/2354] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 14 12:10:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:10:54.571101 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197e055-bd2c-4eab-8e24-7dde40e8cd51 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:10:54.572650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4197e055-bd2c-4eab-8e24-7dde40e8cd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:10:54.603645 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:10:54.610455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ec268-0626-4d2e-99dd-101472d7e090 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:10:54.615334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6768c912-dc7d-469e-aaf8-f8bf2fdbadee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:10:54.615948 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768c912-dc7d-469e-aaf8-f8bf2fdbadee service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:10:54.616236 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768c912-dc7d-469e-aaf8-f8bf2fdbadee service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:10:54.616634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768c912-dc7d-469e-aaf8-f8bf2fdbadee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:10:54.618533 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6768c912-dc7d-469e-aaf8-f8bf2fdbadee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:10:54.649508 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197e055-bd2c-4eab-8e24-7dde40e8cd51 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:10:54.655278 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90e2c6-0886-4126-b7cb-94bd760afe14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1047c7d18c1466d989cd895117cd807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:10:54.656207 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90e2c6-0886-4126-b7cb-94bd760afe14 service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:10:54.657361 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90e2c6-0886-4126-b7cb-94bd760afe14 service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:10:54.657642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90e2c6-0886-4126-b7cb-94bd760afe14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:10:54.666876 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e90e2c6-0886-4126-b7cb-94bd760afe14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:10:54.667384 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:10:54.674422 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e90e2c6-0886-4126-b7cb-94bd760afe14 service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:10:54.678698 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1167/2355] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 14 12:10:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 14 12:10:54.695737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6768c912-dc7d-469e-aaf8-f8bf2fdbadee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:10:54.697169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6768c912-dc7d-469e-aaf8-f8bf2fdbadee service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:10:54.708672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1189/2356] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:10:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:10:55.982388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90e2c6-0886-4126-b7cb-94bd760afe14 service swift] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:10:55.993852 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90e2c6-0886-4126-b7cb-94bd760afe14 service swift] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:10:56.000708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8758ed8-ef53-4a05-9eed-bf5ad38ab803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:10:56.004471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8758ed8-ef53-4a05-9eed-bf5ad38ab803 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:10:56.004471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8758ed8-ef53-4a05-9eed-bf5ad38ab803 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:10:56.004471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8758ed8-ef53-4a05-9eed-bf5ad38ab803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:10:56.004471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b8758ed8-ef53-4a05-9eed-bf5ad38ab803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:10:56.021653 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8758ed8-ef53-4a05-9eed-bf5ad38ab803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:10:56.023228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8758ed8-ef53-4a05-9eed-bf5ad38ab803 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:10:56.039825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1168/2357] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:10:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:11:05.234163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=13 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:11:06.031374 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=27 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:11:08.250457 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6768c912-dc7d-469e-aaf8-f8bf2fdbadee service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:11:08.251992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6768c912-dc7d-469e-aaf8-f8bf2fdbadee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:11:08.266378 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6768c912-dc7d-469e-aaf8-f8bf2fdbadee service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:11:08.271820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e5bd6-abe5-4bd0-a554-bc1ac6953e0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:11:08.277942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e5bd6-abe5-4bd0-a554-bc1ac6953e0a service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:11:08.277942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e5bd6-abe5-4bd0-a554-bc1ac6953e0a service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:11:08.277942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e5bd6-abe5-4bd0-a554-bc1ac6953e0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:11:08.277942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-158e5bd6-abe5-4bd0-a554-bc1ac6953e0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:11:08.333225 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e5bd6-abe5-4bd0-a554-bc1ac6953e0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:11:08.335002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e5bd6-abe5-4bd0-a554-bc1ac6953e0a service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:11:08.344101 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1190/2358] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:11:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:11:12.198845 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8758ed8-ef53-4a05-9eed-bf5ad38ab803 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:11:12.200311 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b8758ed8-ef53-4a05-9eed-bf5ad38ab803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:11:12.215698 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8758ed8-ef53-4a05-9eed-bf5ad38ab803 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:11:12.221764 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e537e-b688-400e-be0f-ae4417a031ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:11:12.222416 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e537e-b688-400e-be0f-ae4417a031ee service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:11:12.222699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e537e-b688-400e-be0f-ae4417a031ee service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:11:12.222978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e537e-b688-400e-be0f-ae4417a031ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:11:12.224779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9b0e537e-b688-400e-be0f-ae4417a031ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:11:12.295114 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e537e-b688-400e-be0f-ae4417a031ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:11:12.296635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e537e-b688-400e-be0f-ae4417a031ee service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:11:12.314002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1169/2359] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:11:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:11:16.827188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e5bd6-abe5-4bd0-a554-bc1ac6953e0a service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:11:16.837783 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e5bd6-abe5-4bd0-a554-bc1ac6953e0a service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:11:16.841439 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581b931-eaaa-4260-bb9e-2f9f91621554 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:11:16.842073 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581b931-eaaa-4260-bb9e-2f9f91621554 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:11:16.842423 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581b931-eaaa-4260-bb9e-2f9f91621554 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:11:16.842872 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581b931-eaaa-4260-bb9e-2f9f91621554 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:11:16.844614 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5581b931-eaaa-4260-bb9e-2f9f91621554 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:11:16.854819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581b931-eaaa-4260-bb9e-2f9f91621554 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:11:16.856238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581b931-eaaa-4260-bb9e-2f9f91621554 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:11:16.864222 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1191/2360] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:11:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:11:17.256997 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a983a79-5a22-4f4d-92d2-a7300f7ab34e None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:11:17.262650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a983a79-5a22-4f4d-92d2-a7300f7ab34e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:11:17.262650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a983a79-5a22-4f4d-92d2-a7300f7ab34e None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:11:17.262650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1170/2361] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 14 12:11:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:11:17.265008 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d905a3a-62c2-43f5-a146-f15132714e2b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:11:17.265380 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d905a3a-62c2-43f5-a146-f15132714e2b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:11:17.265734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d905a3a-62c2-43f5-a146-f15132714e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:11:17.289548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1d905a3a-62c2-43f5-a146-f15132714e2b None None] MFA Rules not processed for user `38b093bc4c1d4ebcbd2a6bb16b70c029`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:11:17.290834 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1d905a3a-62c2-43f5-a146-f15132714e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:11:17.301785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1192/2362] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 14 12:11:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:11:17.313824 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a983a79-5a22-4f4d-92d2-a7300f7ab34e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:11:17.315593 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3a983a79-5a22-4f4d-92d2-a7300f7ab34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:11:17.375629 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a983a79-5a22-4f4d-92d2-a7300f7ab34e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:11:17.379219 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3d1c5-3095-4b45-9fd7-3f661e05d50c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b05d0e65e9645f3810a6d96c8fed9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:11:17.379852 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3d1c5-3095-4b45-9fd7-3f661e05d50c service glance] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:11:17.380135 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3d1c5-3095-4b45-9fd7-3f661e05d50c service glance] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:11:17.380932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3d1c5-3095-4b45-9fd7-3f661e05d50c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:11:17.391525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3d1c5-3095-4b45-9fd7-3f661e05d50c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:11:17.393105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3d1c5-3095-4b45-9fd7-3f661e05d50c service glance] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:11:17.404977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1171/2363] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:11:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:11:17.647158 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee9063-df12-44dd-a2ab-56e6e567cb9a None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:11:17.650995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee9063-df12-44dd-a2ab-56e6e567cb9a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:11:17.650995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee9063-df12-44dd-a2ab-56e6e567cb9a None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:11:17.650995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1193/2364] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 14 12:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:11:17.654269 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030aa365-777f-40e0-861d-9444ef2cb410 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:11:17.654586 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030aa365-777f-40e0-861d-9444ef2cb410 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:11:17.654869 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030aa365-777f-40e0-861d-9444ef2cb410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:11:17.675033 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-030aa365-777f-40e0-861d-9444ef2cb410 None None] MFA Rules not processed for user `13f1840d92d94c3ebd16000a4889c377`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:11:17.676171 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-030aa365-777f-40e0-861d-9444ef2cb410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:11:17.683528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1172/2365] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 14 12:11:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:11:17.696354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee9063-df12-44dd-a2ab-56e6e567cb9a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:11:17.698215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-dbee9063-df12-44dd-a2ab-56e6e567cb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:11:17.712211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee9063-df12-44dd-a2ab-56e6e567cb9a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:11:17.715945 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f8d4f-b899-453a-a4fb-edbf4fbae34e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:11:17.716556 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f8d4f-b899-453a-a4fb-edbf4fbae34e service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:11:17.716839 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f8d4f-b899-453a-a4fb-edbf4fbae34e service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:11:17.717116 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f8d4f-b899-453a-a4fb-edbf4fbae34e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:11:17.725072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f8d4f-b899-453a-a4fb-edbf4fbae34e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:11:17.726451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f8d4f-b899-453a-a4fb-edbf4fbae34e service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:11:17.734793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1194/2366] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:11:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:11:17.824583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c24cf-55e8-41de-a499-d6f6fe7378dd None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:11:17.824878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c24cf-55e8-41de-a499-d6f6fe7378dd None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:11:17.825161 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c24cf-55e8-41de-a499-d6f6fe7378dd None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:11:17.826816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1173/2367] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 14 12:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:11:17.834118 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409b4ad-eeda-42c4-9a6b-27d621c95a8a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:11:17.834647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409b4ad-eeda-42c4-9a6b-27d621c95a8a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:11:17.834766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409b4ad-eeda-42c4-9a6b-27d621c95a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:11:17.854695 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-5409b4ad-eeda-42c4-9a6b-27d621c95a8a None None] MFA Rules not processed for user `38b093bc4c1d4ebcbd2a6bb16b70c029`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:11:17.856108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5409b4ad-eeda-42c4-9a6b-27d621c95a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:11:17.865510 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1195/2368] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 14 12:11:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:11:17.876848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c24cf-55e8-41de-a499-d6f6fe7378dd None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:11:17.888078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c24cf-55e8-41de-a499-d6f6fe7378dd None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:11:17.891787 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d051a5-8e87-4027-ab29-089967e78545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:11:17.892382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d051a5-8e87-4027-ab29-089967e78545 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:11:17.894747 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d051a5-8e87-4027-ab29-089967e78545 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:11:17.894747 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d051a5-8e87-4027-ab29-089967e78545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:11:17.903709 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d051a5-8e87-4027-ab29-089967e78545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:11:17.905139 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d051a5-8e87-4027-ab29-089967e78545 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:11:17.916558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1174/2369] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:11:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:11:17.979197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409b4ad-eeda-42c4-9a6b-27d621c95a8a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:11:17.980649 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5409b4ad-eeda-42c4-9a6b-27d621c95a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:11:18.055089 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409b4ad-eeda-42c4-9a6b-27d621c95a8a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:11:18.060154 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b9151-c1f8-4ad5-ba61-692f8284a972 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca10dd7ed714edb8b87017deecc41b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:11:18.060757 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b9151-c1f8-4ad5-ba61-692f8284a972 service placement] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:11:18.061039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b9151-c1f8-4ad5-ba61-692f8284a972 service placement] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:11:18.061309 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b9151-c1f8-4ad5-ba61-692f8284a972 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:11:18.063067 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b90b9151-c1f8-4ad5-ba61-692f8284a972 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:11:18.074295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b9151-c1f8-4ad5-ba61-692f8284a972 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:11:18.075867 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b9151-c1f8-4ad5-ba61-692f8284a972 service placement] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:11:18.085482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1196/2370] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:11:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:11:20.873115 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603b833-9d16-434e-a20e-584b308a42d8 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:11:20.873650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603b833-9d16-434e-a20e-584b308a42d8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:11:20.873650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603b833-9d16-434e-a20e-584b308a42d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:11:20.895669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-0603b833-9d16-434e-a20e-584b308a42d8 None None] MFA Rules not processed for user `8fe575351bdc456682db592f0c723ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:11:20.896902 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0603b833-9d16-434e-a20e-584b308a42d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:11:20.956051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1175/2371] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:11:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:11:24.777122 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fae6b1-a9e3-4360-ba2e-5c041893c86a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:11:24.777700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fae6b1-a9e3-4360-ba2e-5c041893c86a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:11:24.777700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fae6b1-a9e3-4360-ba2e-5c041893c86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:11:24.797722 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-b8fae6b1-a9e3-4360-ba2e-5c041893c86a None None] MFA Rules not processed for user `d76a164ed7e14d9eb2db016b68a84d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:11:24.799062 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b8fae6b1-a9e3-4360-ba2e-5c041893c86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:11:24.807652 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1197/2372] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 14 12:11:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:11:24.838952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603b833-9d16-434e-a20e-584b308a42d8 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:11:24.852325 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603b833-9d16-434e-a20e-584b308a42d8 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:11:24.858809 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70810de-a504-4f0a-9cb6-6c32c4616600 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1047c7d18c1466d989cd895117cd807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:11:24.859469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70810de-a504-4f0a-9cb6-6c32c4616600 service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:11:24.859818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70810de-a504-4f0a-9cb6-6c32c4616600 service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:11:24.860157 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70810de-a504-4f0a-9cb6-6c32c4616600 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:11:24.869736 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70810de-a504-4f0a-9cb6-6c32c4616600 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:11:24.871408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70810de-a504-4f0a-9cb6-6c32c4616600 service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:11:24.879772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1176/2373] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 14 12:11:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 12:11:34.792255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=81 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:11:34.876445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=94 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:11:46.474551 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fae6b1-a9e3-4360-ba2e-5c041893c86a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:11:46.485222 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fae6b1-a9e3-4360-ba2e-5c041893c86a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:11:46.488849 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91fede0-ec55-461b-8f4d-2252488cb969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:11:46.489488 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fede0-ec55-461b-8f4d-2252488cb969 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:11:46.489734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fede0-ec55-461b-8f4d-2252488cb969 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:11:46.490044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fede0-ec55-461b-8f4d-2252488cb969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:11:46.491955 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c91fede0-ec55-461b-8f4d-2252488cb969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:11:46.503423 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91fede0-ec55-461b-8f4d-2252488cb969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:11:46.504844 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91fede0-ec55-461b-8f4d-2252488cb969 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:11:46.516907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1198/2374] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:11:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:11:47.585138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70810de-a504-4f0a-9cb6-6c32c4616600 service swift] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:11:47.598456 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70810de-a504-4f0a-9cb6-6c32c4616600 service swift] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:11:47.604754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9060b-d880-4c4b-916c-93b85816f4dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:11:47.605428 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9060b-d880-4c4b-916c-93b85816f4dc service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:11:47.605727 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9060b-d880-4c4b-916c-93b85816f4dc service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:11:47.606032 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9060b-d880-4c4b-916c-93b85816f4dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:11:47.611103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ccc9060b-d880-4c4b-916c-93b85816f4dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:11:47.623950 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9060b-d880-4c4b-916c-93b85816f4dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:11:47.625701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9060b-d880-4c4b-916c-93b85816f4dc service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:11:47.638582 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1177/2375] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:11:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:11:56.517547 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=6 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:11:57.631892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=6 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:12:01.913903 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91fede0-ec55-461b-8f4d-2252488cb969 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:12:01.929849 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91fede0-ec55-461b-8f4d-2252488cb969 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:12:01.936475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52915a2e-bfeb-4567-a8ad-f998866d1e39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:12:01.937085 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52915a2e-bfeb-4567-a8ad-f998866d1e39 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:12:01.937405 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52915a2e-bfeb-4567-a8ad-f998866d1e39 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:12:01.937637 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52915a2e-bfeb-4567-a8ad-f998866d1e39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:12:01.939529 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-52915a2e-bfeb-4567-a8ad-f998866d1e39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:12:01.953931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52915a2e-bfeb-4567-a8ad-f998866d1e39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:12:01.955565 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52915a2e-bfeb-4567-a8ad-f998866d1e39 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:12:01.966371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1199/2376] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:12:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:12:06.448692 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f0b85-6b46-41a0-bf2e-2a2011868895 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:12:06.455023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f0b85-6b46-41a0-bf2e-2a2011868895 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:12:06.455023 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f0b85-6b46-41a0-bf2e-2a2011868895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:12:06.472108 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ba1f0b85-6b46-41a0-bf2e-2a2011868895 None None] MFA Rules not processed for user `d76a164ed7e14d9eb2db016b68a84d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:12:06.473649 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ba1f0b85-6b46-41a0-bf2e-2a2011868895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:12:06.481782 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1178/2377] 10.222.0.62 () {58 vars in 967 bytes} [Wed Sep 14 12:12:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:12:06.496081 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52915a2e-bfeb-4567-a8ad-f998866d1e39 service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:12:06.508535 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52915a2e-bfeb-4567-a8ad-f998866d1e39 service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:12:06.515501 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe84bdc-b710-4e2e-8b82-e871bb86991c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1047c7d18c1466d989cd895117cd807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:12:06.516340 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe84bdc-b710-4e2e-8b82-e871bb86991c service swift] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:12:06.516786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe84bdc-b710-4e2e-8b82-e871bb86991c service swift] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:12:06.517220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe84bdc-b710-4e2e-8b82-e871bb86991c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:12:06.529403 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe84bdc-b710-4e2e-8b82-e871bb86991c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:12:06.531405 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe84bdc-b710-4e2e-8b82-e871bb86991c service swift] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:12:06.541374 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1200/2378] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 14 12:12:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 12:12:07.698132 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72534dd7-b6dc-4c9d-a072-9558209f55c0 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:12:07.699193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72534dd7-b6dc-4c9d-a072-9558209f55c0 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:12:07.699193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72534dd7-b6dc-4c9d-a072-9558209f55c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:12:07.719582 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-72534dd7-b6dc-4c9d-a072-9558209f55c0 None None] MFA Rules not processed for user `31f3271adc36406fb303fb169020e740`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:12:07.720874 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-72534dd7-b6dc-4c9d-a072-9558209f55c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:12:07.778862 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1179/2379] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:12:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:12:16.537529 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=12 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:12:17.771268 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=28 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:12:51.519310 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe84bdc-b710-4e2e-8b82-e871bb86991c service swift] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:12:51.563825 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe84bdc-b710-4e2e-8b82-e871bb86991c service swift] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:12:51.570260 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eadd24-18b8-4ad6-8342-018dc4c2e20d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:12:51.570866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eadd24-18b8-4ad6-8342-018dc4c2e20d service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:12:51.571151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eadd24-18b8-4ad6-8342-018dc4c2e20d service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:12:51.571455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eadd24-18b8-4ad6-8342-018dc4c2e20d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:12:51.573239 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-89eadd24-18b8-4ad6-8342-018dc4c2e20d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:12:51.641668 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eadd24-18b8-4ad6-8342-018dc4c2e20d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:12:51.643274 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eadd24-18b8-4ad6-8342-018dc4c2e20d service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:12:51.652793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1201/2380] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:12:51 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:13:01.648074 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=46 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:13:15.095826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72534dd7-b6dc-4c9d-a072-9558209f55c0 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:13:15.098529 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-72534dd7-b6dc-4c9d-a072-9558209f55c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:13:15.164885 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72534dd7-b6dc-4c9d-a072-9558209f55c0 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:13:15.169464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637a559-2d6c-48eb-989d-56377a7cb08d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca10dd7ed714edb8b87017deecc41b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:13:15.170117 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637a559-2d6c-48eb-989d-56377a7cb08d service placement] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:13:15.170395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637a559-2d6c-48eb-989d-56377a7cb08d service placement] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:13:15.170668 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637a559-2d6c-48eb-989d-56377a7cb08d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:13:15.172400 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d637a559-2d6c-48eb-989d-56377a7cb08d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:13:15.186529 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637a559-2d6c-48eb-989d-56377a7cb08d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:13:15.188239 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637a559-2d6c-48eb-989d-56377a7cb08d service placement] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:13:15.203154 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1180/2381] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:13:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:13:15.254232 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eadd24-18b8-4ad6-8342-018dc4c2e20d service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:13:15.255852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-89eadd24-18b8-4ad6-8342-018dc4c2e20d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:13:15.376365 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eadd24-18b8-4ad6-8342-018dc4c2e20d service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:13:15.381325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa53743-53c3-4ecf-be80-228e57f1abd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:13:15.381976 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa53743-53c3-4ecf-be80-228e57f1abd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:13:15.382262 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa53743-53c3-4ecf-be80-228e57f1abd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:13:15.382541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa53743-53c3-4ecf-be80-228e57f1abd5 admin admin] PATH_INFO: `/v3/users/8fe575351bdc456682db592f0c723ea6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:13:15.383693 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa53743-53c3-4ecf-be80-228e57f1abd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fe575351bdc456682db592f0c723ea6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:13:15.385653 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa53743-53c3-4ecf-be80-228e57f1abd5 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:13:15.535081 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-daa53743-53c3-4ecf-be80-228e57f1abd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fe575351bdc456682db592f0c723ea6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:13:15.551860 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-daa53743-53c3-4ecf-be80-228e57f1abd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fe575351bdc456682db592f0c723ea6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:13:15.560492 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-daa53743-53c3-4ecf-be80-228e57f1abd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fe575351bdc456682db592f0c723ea6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:13:15.627764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-daa53743-53c3-4ecf-be80-228e57f1abd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fe575351bdc456682db592f0c723ea6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:13:15.628324 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-daa53743-53c3-4ecf-be80-228e57f1abd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fe575351bdc456682db592f0c723ea6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:13:15.634994 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1202/2382] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:13:15 2022] DELETE /identity/v3/users/8fe575351bdc456682db592f0c723ea6 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:13:15.911080 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637a559-2d6c-48eb-989d-56377a7cb08d service placement] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:13:15.924982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637a559-2d6c-48eb-989d-56377a7cb08d service placement] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:13:15.993116 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15017c-632d-45f2-80fb-ecfe85d4a8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:13:15.993962 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15017c-632d-45f2-80fb-ecfe85d4a8ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:13:15.994357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15017c-632d-45f2-80fb-ecfe85d4a8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:13:15.994743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15017c-632d-45f2-80fb-ecfe85d4a8ce admin admin] PATH_INFO: `/v3/projects/1488f00baa934f2e9b910644a179c755` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:13:15.996137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15017c-632d-45f2-80fb-ecfe85d4a8ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=1488f00baa934f2e9b910644a179c755)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:13:15.998237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15017c-632d-45f2-80fb-ecfe85d4a8ce admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:13:16.191670 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6c15017c-632d-45f2-80fb-ecfe85d4a8ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1488f00baa934f2e9b910644a179c755', 'request_id': 'req-6c15017c-632d-45f2-80fb-ecfe85d4a8ce'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:13:16.302500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6c15017c-632d-45f2-80fb-ecfe85d4a8ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1488f00baa934f2e9b910644a179c755', 'request_id': 'req-6c15017c-632d-45f2-80fb-ecfe85d4a8ce'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:13:16.310449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6c15017c-632d-45f2-80fb-ecfe85d4a8ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1488f00baa934f2e9b910644a179c755', 'request_id': 'req-6c15017c-632d-45f2-80fb-ecfe85d4a8ce'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:13:16.326126 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6c15017c-632d-45f2-80fb-ecfe85d4a8ce admin admin] The token cache is being invalidate because project 1488f00baa934f2e9b910644a179c755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:13:16.326447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6c15017c-632d-45f2-80fb-ecfe85d4a8ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:13:16.327176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6c15017c-632d-45f2-80fb-ecfe85d4a8ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:13:16.328683 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1181/2383] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:13:15 2022] DELETE /identity/v3/projects/1488f00baa934f2e9b910644a179c755 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:13:21.053146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc1403-6e19-472e-ba7a-8ed8586a4b15 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:13:21.053914 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc1403-6e19-472e-ba7a-8ed8586a4b15 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:13:21.054171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc1403-6e19-472e-ba7a-8ed8586a4b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:13:21.078554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-95bc1403-6e19-472e-ba7a-8ed8586a4b15 None None] MFA Rules not processed for user `4bd4125ff6bb400bae1b94df7d2cfa17`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:13:21.079791 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-95bc1403-6e19-472e-ba7a-8ed8586a4b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:13:21.146870 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:13:21.160907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1203/2384] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:13:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:13:26.323741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:13:31.253872 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:13:55.549607 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8133d-8a9f-4472-9a40-171663e59b42 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:13:55.550442 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8133d-8a9f-4472-9a40-171663e59b42 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:13:55.550607 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8133d-8a9f-4472-9a40-171663e59b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:13:55.569750 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-08f8133d-8a9f-4472-9a40-171663e59b42 None None] MFA Rules not processed for user `28e44067e07d4501a840108f4dcf053c`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:13:55.571001 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-08f8133d-8a9f-4472-9a40-171663e59b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:13:55.626677 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1182/2385] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:13:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:13:55.936614 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc1403-6e19-472e-ba7a-8ed8586a4b15 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:13:55.938043 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-95bc1403-6e19-472e-ba7a-8ed8586a4b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:13:56.019872 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc1403-6e19-472e-ba7a-8ed8586a4b15 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:13:56.024065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2801f-7d33-45cb-8031-e95355c858a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:13:56.024932 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2801f-7d33-45cb-8031-e95355c858a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:13:56.025210 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2801f-7d33-45cb-8031-e95355c858a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:13:56.025486 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2801f-7d33-45cb-8031-e95355c858a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:13:56.027237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-50d2801f-7d33-45cb-8031-e95355c858a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:13:56.044334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2801f-7d33-45cb-8031-e95355c858a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:13:56.045774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2801f-7d33-45cb-8031-e95355c858a1 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:13:56.056741 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1204/2386] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:13:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:14:05.619997 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=25 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:14:06.050340 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=35 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:14:21.298827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8133d-8a9f-4472-9a40-171663e59b42 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:14:21.304804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-08f8133d-8a9f-4472-9a40-171663e59b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:14:21.410422 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8133d-8a9f-4472-9a40-171663e59b42 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:14:21.417066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51db4a-f27d-488b-a22d-2df2f28a0a50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:14:21.421889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51db4a-f27d-488b-a22d-2df2f28a0a50 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:14:21.421889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51db4a-f27d-488b-a22d-2df2f28a0a50 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:14:21.421889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51db4a-f27d-488b-a22d-2df2f28a0a50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:14:21.421889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4d51db4a-f27d-488b-a22d-2df2f28a0a50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:14:21.498477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51db4a-f27d-488b-a22d-2df2f28a0a50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:14:21.500208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51db4a-f27d-488b-a22d-2df2f28a0a50 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:14:21.509763 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1183/2387] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:14:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:14:26.207675 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2801f-7d33-45cb-8031-e95355c858a1 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:14:26.209712 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-50d2801f-7d33-45cb-8031-e95355c858a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:14:26.316686 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2801f-7d33-45cb-8031-e95355c858a1 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:14:26.322689 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f1509-ea0b-489c-9bbe-67699520ab99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca10dd7ed714edb8b87017deecc41b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:14:26.323361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f1509-ea0b-489c-9bbe-67699520ab99 service placement] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:14:26.323641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f1509-ea0b-489c-9bbe-67699520ab99 service placement] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:14:26.323906 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f1509-ea0b-489c-9bbe-67699520ab99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:14:26.325735 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-259f1509-ea0b-489c-9bbe-67699520ab99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:14:26.338601 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f1509-ea0b-489c-9bbe-67699520ab99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:14:26.340199 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f1509-ea0b-489c-9bbe-67699520ab99 service placement] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:14:26.350987 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1205/2388] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:14:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:14:31.505249 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=68 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:14:36.346401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=41 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:14:41.389479 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03b733-8fa4-4717-9882-5e827ddd966f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:14:41.390061 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03b733-8fa4-4717-9882-5e827ddd966f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:14:41.390175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03b733-8fa4-4717-9882-5e827ddd966f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:14:41.412949 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ad03b733-8fa4-4717-9882-5e827ddd966f None None] MFA Rules not processed for user `31f3271adc36406fb303fb169020e740`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:14:41.425504 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ad03b733-8fa4-4717-9882-5e827ddd966f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:14:41.425504 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1184/2389] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:14:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:14:50.977993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800623f4-865d-49fd-b104-babb28d522cc None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:14:50.978550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800623f4-865d-49fd-b104-babb28d522cc None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:14:50.978550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800623f4-865d-49fd-b104-babb28d522cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:14:50.998495 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-800623f4-865d-49fd-b104-babb28d522cc None None] MFA Rules not processed for user `2ba1860ce5504c27816a504776463d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:14:51.000220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-800623f4-865d-49fd-b104-babb28d522cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:14:51.062256 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1206/2390] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:14:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:14:51.406649 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:15:01.054605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=25 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:15:07.487364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03b733-8fa4-4717-9882-5e827ddd966f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:07.488968 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ad03b733-8fa4-4717-9882-5e827ddd966f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:07.506961 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03b733-8fa4-4717-9882-5e827ddd966f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:07.515261 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49e866-e7f9-4fde-89ce-b5042b7c3596 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:07.516294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49e866-e7f9-4fde-89ce-b5042b7c3596 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:07.516294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49e866-e7f9-4fde-89ce-b5042b7c3596 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:07.516497 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49e866-e7f9-4fde-89ce-b5042b7c3596 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:07.518234 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8e49e866-e7f9-4fde-89ce-b5042b7c3596 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:07.530996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49e866-e7f9-4fde-89ce-b5042b7c3596 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:07.532566 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49e866-e7f9-4fde-89ce-b5042b7c3596 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:07.544351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1185/2391] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:15:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:15:12.468226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800623f4-865d-49fd-b104-babb28d522cc None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:12.469670 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-800623f4-865d-49fd-b104-babb28d522cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:12.481773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800623f4-865d-49fd-b104-babb28d522cc None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:12.485330 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edee9fe-fcaf-4b4d-b009-a85268f390ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:12.485963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edee9fe-fcaf-4b4d-b009-a85268f390ac service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:12.486252 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edee9fe-fcaf-4b4d-b009-a85268f390ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:12.486531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edee9fe-fcaf-4b4d-b009-a85268f390ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:12.488362 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1edee9fe-fcaf-4b4d-b009-a85268f390ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:12.552928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edee9fe-fcaf-4b4d-b009-a85268f390ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:12.554437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edee9fe-fcaf-4b4d-b009-a85268f390ac service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:12.564933 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1207/2392] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:15:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:15:16.705898 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49e866-e7f9-4fde-89ce-b5042b7c3596 service nova] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:16.712078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8e49e866-e7f9-4fde-89ce-b5042b7c3596 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:16.723248 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49e866-e7f9-4fde-89ce-b5042b7c3596 service nova] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:16.728072 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a328796a-f3a0-400a-92b7-1b8edd1290ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:16.728696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328796a-f3a0-400a-92b7-1b8edd1290ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:16.728980 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328796a-f3a0-400a-92b7-1b8edd1290ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:16.729257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328796a-f3a0-400a-92b7-1b8edd1290ce admin admin] PATH_INFO: `/v3/users/4bd4125ff6bb400bae1b94df7d2cfa17` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:16.730410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a328796a-f3a0-400a-92b7-1b8edd1290ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bd4125ff6bb400bae1b94df7d2cfa17)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:16.732361 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a328796a-f3a0-400a-92b7-1b8edd1290ce admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:16.971111 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a328796a-f3a0-400a-92b7-1b8edd1290ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bd4125ff6bb400bae1b94df7d2cfa17'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:16.983133 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a328796a-f3a0-400a-92b7-1b8edd1290ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bd4125ff6bb400bae1b94df7d2cfa17'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:16.990100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a328796a-f3a0-400a-92b7-1b8edd1290ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bd4125ff6bb400bae1b94df7d2cfa17'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:17.084453 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a328796a-f3a0-400a-92b7-1b8edd1290ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bd4125ff6bb400bae1b94df7d2cfa17'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:17.085086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a328796a-f3a0-400a-92b7-1b8edd1290ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bd4125ff6bb400bae1b94df7d2cfa17'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:17.091805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1186/2393] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:15:16 2022] DELETE /identity/v3/users/4bd4125ff6bb400bae1b94df7d2cfa17 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:15:17.358458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edee9fe-fcaf-4b4d-b009-a85268f390ac service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:17.369387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edee9fe-fcaf-4b4d-b009-a85268f390ac service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:17.435837 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1528179-c385-4a5b-95af-6199b6fc0520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:17.436502 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1528179-c385-4a5b-95af-6199b6fc0520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:17.436789 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1528179-c385-4a5b-95af-6199b6fc0520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:17.437068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1528179-c385-4a5b-95af-6199b6fc0520 admin admin] PATH_INFO: `/v3/projects/748b7c08460b4f718dfff67275d8e4e4` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:17.438754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1528179-c385-4a5b-95af-6199b6fc0520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=748b7c08460b4f718dfff67275d8e4e4)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:17.440820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1528179-c385-4a5b-95af-6199b6fc0520 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:17.553302 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e1528179-c385-4a5b-95af-6199b6fc0520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '748b7c08460b4f718dfff67275d8e4e4', 'request_id': 'req-e1528179-c385-4a5b-95af-6199b6fc0520'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:17.637189 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e1528179-c385-4a5b-95af-6199b6fc0520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '748b7c08460b4f718dfff67275d8e4e4', 'request_id': 'req-e1528179-c385-4a5b-95af-6199b6fc0520'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:17.645147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e1528179-c385-4a5b-95af-6199b6fc0520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '748b7c08460b4f718dfff67275d8e4e4', 'request_id': 'req-e1528179-c385-4a5b-95af-6199b6fc0520'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:17.656196 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e1528179-c385-4a5b-95af-6199b6fc0520 admin admin] The token cache is being invalidate because project 748b7c08460b4f718dfff67275d8e4e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:15:17.656485 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e1528179-c385-4a5b-95af-6199b6fc0520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:17.657137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e1528179-c385-4a5b-95af-6199b6fc0520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:17.658662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1208/2394] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:15:17 2022] DELETE /identity/v3/projects/748b7c08460b4f718dfff67275d8e4e4 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:15:17.665103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d61f21-1c07-4b7e-b93b-01d2147515e3 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:17.665399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d61f21-1c07-4b7e-b93b-01d2147515e3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:17.665675 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d61f21-1c07-4b7e-b93b-01d2147515e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:17.724369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e9d61f21-1c07-4b7e-b93b-01d2147515e3 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:15:17.726359 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e9d61f21-1c07-4b7e-b93b-01d2147515e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:17.785494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1187/2395] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:15:17.794126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ce135-cfa8-4ea5-88ab-cd8a87840e38 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:17.794417 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ce135-cfa8-4ea5-88ab-cd8a87840e38 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:17.794730 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ce135-cfa8-4ea5-88ab-cd8a87840e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:17.815035 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1d4ce135-cfa8-4ea5-88ab-cd8a87840e38 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:15:17.816326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1d4ce135-cfa8-4ea5-88ab-cd8a87840e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:17.824343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1209/2396] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:15:17.829816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d61f21-1c07-4b7e-b93b-01d2147515e3 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:17.839721 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d61f21-1c07-4b7e-b93b-01d2147515e3 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:17.844781 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bbe1b-3798-4055-b170-86234bfdef3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:17.845431 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bbe1b-3798-4055-b170-86234bfdef3a admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:17.851392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bbe1b-3798-4055-b170-86234bfdef3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:17.851392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bbe1b-3798-4055-b170-86234bfdef3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:17.851392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bbe1b-3798-4055-b170-86234bfdef3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:15:17.851392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bbe1b-3798-4055-b170-86234bfdef3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:17.851392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bbe1b-3798-4055-b170-86234bfdef3a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:17.861703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1188/2397] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:15:17.866235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ce135-cfa8-4ea5-88ab-cd8a87840e38 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:17.876621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ce135-cfa8-4ea5-88ab-cd8a87840e38 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:17.881400 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2b406-ead3-4e47-a99b-21ccfa4e95ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:17.882042 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2b406-ead3-4e47-a99b-21ccfa4e95ee admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:17.882333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2b406-ead3-4e47-a99b-21ccfa4e95ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:17.882622 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2b406-ead3-4e47-a99b-21ccfa4e95ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:17.883260 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd2b406-ead3-4e47-a99b-21ccfa4e95ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:17.885185 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd2b406-ead3-4e47-a99b-21ccfa4e95ee admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:17.902113 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:15:18.005687 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1210/2398] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:15:17 2022] POST /identity/v3/projects => generated 367 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:15:18.010099 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bbe1b-3798-4055-b170-86234bfdef3a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:18.020587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bbe1b-3798-4055-b170-86234bfdef3a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:18.032593 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:15:18.078547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c07e4e-ea53-4d71-a1c5-022a5faf8788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:18.079151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c07e4e-ea53-4d71-a1c5-022a5faf8788 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:18.079432 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c07e4e-ea53-4d71-a1c5-022a5faf8788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:18.079704 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c07e4e-ea53-4d71-a1c5-022a5faf8788 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:18.080328 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c07e4e-ea53-4d71-a1c5-022a5faf8788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:18.082269 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c07e4e-ea53-4d71-a1c5-022a5faf8788 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:18.174335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1189/2399] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:15:18 2022] POST /identity/v3/users => generated 284 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:15:18.178571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2b406-ead3-4e47-a99b-21ccfa4e95ee admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:18.189771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2b406-ead3-4e47-a99b-21ccfa4e95ee admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:18.194689 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df2a7e-b80c-4886-8844-0866e6745c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:18.195307 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df2a7e-b80c-4886-8844-0866e6745c84 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:18.195573 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df2a7e-b80c-4886-8844-0866e6745c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:18.195849 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df2a7e-b80c-4886-8844-0866e6745c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:18.196487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df2a7e-b80c-4886-8844-0866e6745c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:18.198269 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df2a7e-b80c-4886-8844-0866e6745c84 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:18.208746 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1211/2400] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:15:18.213173 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c07e4e-ea53-4d71-a1c5-022a5faf8788 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:18.223658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c07e4e-ea53-4d71-a1c5-022a5faf8788 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:18.228310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055c9a0-9507-4c09-95fc-ee5accd5e9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:18.228932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055c9a0-9507-4c09-95fc-ee5accd5e9bf admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:18.229221 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055c9a0-9507-4c09-95fc-ee5accd5e9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:18.229562 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055c9a0-9507-4c09-95fc-ee5accd5e9bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:18.230247 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055c9a0-9507-4c09-95fc-ee5accd5e9bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:18.231903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055c9a0-9507-4c09-95fc-ee5accd5e9bf admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:18.243928 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1190/2401] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:15:18.247497 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df2a7e-b80c-4886-8844-0866e6745c84 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:18.258204 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df2a7e-b80c-4886-8844-0866e6745c84 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:18.262854 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a7db9-5f7f-4907-a82c-36634b385122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:18.263498 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a7db9-5f7f-4907-a82c-36634b385122 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:18.263781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a7db9-5f7f-4907-a82c-36634b385122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:18.264062 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a7db9-5f7f-4907-a82c-36634b385122 admin admin] PATH_INFO: `/v3/projects/55ec1780b31041eba47661a6e6c523dd/users/267a404508fd483ea0e636319e022a8b/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:18.279041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a7db9-5f7f-4907-a82c-36634b385122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55ec1780b31041eba47661a6e6c523dd, user_id=267a404508fd483ea0e636319e022a8b, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:18.282375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a7db9-5f7f-4907-a82c-36634b385122 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:18.368689 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1212/2402] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:15:18 2022] PUT /identity/v3/projects/55ec1780b31041eba47661a6e6c523dd/users/267a404508fd483ea0e636319e022a8b/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:15:18.379148 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15267c-2516-4f5b-8b0c-efdb6b4a5a08 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:18.379511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15267c-2516-4f5b-8b0c-efdb6b4a5a08 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:18.379796 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15267c-2516-4f5b-8b0c-efdb6b4a5a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:18.409780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1c15267c-2516-4f5b-8b0c-efdb6b4a5a08 None None] MFA Rules not processed for user `267a404508fd483ea0e636319e022a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:15:18.411058 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1c15267c-2516-4f5b-8b0c-efdb6b4a5a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:18.476429 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1191/2403] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:15:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:15:18.488513 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a7db9-5f7f-4907-a82c-36634b385122 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:18.490328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ad5a7db9-5f7f-4907-a82c-36634b385122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:18.570623 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a7db9-5f7f-4907-a82c-36634b385122 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:18.574293 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6cb208-3345-4cd3-ae91-e307ac15fb8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:18.574901 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cb208-3345-4cd3-ae91-e307ac15fb8c service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:18.575187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cb208-3345-4cd3-ae91-e307ac15fb8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:18.575465 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cb208-3345-4cd3-ae91-e307ac15fb8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:18.583864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6cb208-3345-4cd3-ae91-e307ac15fb8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:18.585279 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6cb208-3345-4cd3-ae91-e307ac15fb8c service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:18.594770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1213/2404] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:15:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:15:28.473304 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=59,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:15:28.590169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=55,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:15:31.551827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15267c-2516-4f5b-8b0c-efdb6b4a5a08 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:31.613446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15267c-2516-4f5b-8b0c-efdb6b4a5a08 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:31.617935 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606598e9-ba00-4a26-9546-87a867c1c643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:31.618559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606598e9-ba00-4a26-9546-87a867c1c643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:31.618841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606598e9-ba00-4a26-9546-87a867c1c643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:31.619117 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606598e9-ba00-4a26-9546-87a867c1c643 admin admin] PATH_INFO: `/v3/users/267a404508fd483ea0e636319e022a8b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:31.620268 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606598e9-ba00-4a26-9546-87a867c1c643 admin admin] RBAC: Authorizing `identity:delete_user(user_id=267a404508fd483ea0e636319e022a8b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:31.622222 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606598e9-ba00-4a26-9546-87a867c1c643 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:31.832779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-606598e9-ba00-4a26-9546-87a867c1c643 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '267a404508fd483ea0e636319e022a8b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:31.846241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-606598e9-ba00-4a26-9546-87a867c1c643 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '267a404508fd483ea0e636319e022a8b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:31.853341 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-606598e9-ba00-4a26-9546-87a867c1c643 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '267a404508fd483ea0e636319e022a8b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:31.944911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-606598e9-ba00-4a26-9546-87a867c1c643 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '267a404508fd483ea0e636319e022a8b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:31.945546 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-606598e9-ba00-4a26-9546-87a867c1c643 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '267a404508fd483ea0e636319e022a8b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:31.952184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1192/2405] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:15:31 2022] DELETE /identity/v3/users/267a404508fd483ea0e636319e022a8b => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:15:31.956679 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976e191-fe1b-4048-865a-bdc80b41e674 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:31.957186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976e191-fe1b-4048-865a-bdc80b41e674 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:31.957301 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976e191-fe1b-4048-865a-bdc80b41e674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:31.977249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7976e191-fe1b-4048-865a-bdc80b41e674 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:15:31.978501 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7976e191-fe1b-4048-865a-bdc80b41e674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:32.039382 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1214/2406] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:15:32.049612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606598e9-ba00-4a26-9546-87a867c1c643 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:32.051083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-606598e9-ba00-4a26-9546-87a867c1c643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:32.107960 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606598e9-ba00-4a26-9546-87a867c1c643 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:32.112438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fd508-4619-45b2-8d3c-231fdddd1280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:32.113033 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fd508-4619-45b2-8d3c-231fdddd1280 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:32.113315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fd508-4619-45b2-8d3c-231fdddd1280 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:32.113589 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fd508-4619-45b2-8d3c-231fdddd1280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:32.122566 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fd508-4619-45b2-8d3c-231fdddd1280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:32.123970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fd508-4619-45b2-8d3c-231fdddd1280 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:32.133345 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1193/2407] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:15:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:15:32.526505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976e191-fe1b-4048-865a-bdc80b41e674 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:32.528300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7976e191-fe1b-4048-865a-bdc80b41e674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:32.541740 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976e191-fe1b-4048-865a-bdc80b41e674 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:32.546523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b4fbf-c72f-4771-94fd-1860a3444061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:32.547500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b4fbf-c72f-4771-94fd-1860a3444061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:32.547500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b4fbf-c72f-4771-94fd-1860a3444061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:32.547829 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b4fbf-c72f-4771-94fd-1860a3444061 admin admin] PATH_INFO: `/v3/projects/55ec1780b31041eba47661a6e6c523dd` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:32.549037 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b4fbf-c72f-4771-94fd-1860a3444061 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55ec1780b31041eba47661a6e6c523dd)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:32.551397 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b4fbf-c72f-4771-94fd-1860a3444061 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:32.669676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-875b4fbf-c72f-4771-94fd-1860a3444061 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55ec1780b31041eba47661a6e6c523dd', 'request_id': 'req-875b4fbf-c72f-4771-94fd-1860a3444061'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:32.740995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-875b4fbf-c72f-4771-94fd-1860a3444061 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55ec1780b31041eba47661a6e6c523dd', 'request_id': 'req-875b4fbf-c72f-4771-94fd-1860a3444061'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:32.747809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-875b4fbf-c72f-4771-94fd-1860a3444061 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55ec1780b31041eba47661a6e6c523dd', 'request_id': 'req-875b4fbf-c72f-4771-94fd-1860a3444061'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:32.759230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-875b4fbf-c72f-4771-94fd-1860a3444061 admin admin] The token cache is being invalidate because project 55ec1780b31041eba47661a6e6c523dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:15:32.759524 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-875b4fbf-c72f-4771-94fd-1860a3444061 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:32.760183 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-875b4fbf-c72f-4771-94fd-1860a3444061 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:15:32.761683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1215/2408] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:15:32 2022] DELETE /identity/v3/projects/55ec1780b31041eba47661a6e6c523dd => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:15:32.767455 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04163b6-114f-4277-ab1a-ecfc38e7ece9 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:32.767930 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04163b6-114f-4277-ab1a-ecfc38e7ece9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:32.768071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04163b6-114f-4277-ab1a-ecfc38e7ece9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:32.788174 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e04163b6-114f-4277-ab1a-ecfc38e7ece9 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:15:32.789426 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e04163b6-114f-4277-ab1a-ecfc38e7ece9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:32.807102 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:15:32.854349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1194/2409] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:15:32.862525 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817d2c8-082c-4a8a-822e-048fc30b1627 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:32.862809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817d2c8-082c-4a8a-822e-048fc30b1627 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:32.863087 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817d2c8-082c-4a8a-822e-048fc30b1627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:32.889186 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9817d2c8-082c-4a8a-822e-048fc30b1627 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:15:32.890463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9817d2c8-082c-4a8a-822e-048fc30b1627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:32.900266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1216/2410] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:15:32.916793 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04163b6-114f-4277-ab1a-ecfc38e7ece9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:32.924584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04163b6-114f-4277-ab1a-ecfc38e7ece9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:32.929889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108ccd9-80d5-4968-84ab-7341b8f4078e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:32.931928 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108ccd9-80d5-4968-84ab-7341b8f4078e admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:32.931928 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108ccd9-80d5-4968-84ab-7341b8f4078e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:32.931928 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108ccd9-80d5-4968-84ab-7341b8f4078e admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:32.931928 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108ccd9-80d5-4968-84ab-7341b8f4078e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:15:32.932369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108ccd9-80d5-4968-84ab-7341b8f4078e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:32.933604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108ccd9-80d5-4968-84ab-7341b8f4078e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:32.947447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1195/2411] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:15:32.952761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817d2c8-082c-4a8a-822e-048fc30b1627 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:32.963820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817d2c8-082c-4a8a-822e-048fc30b1627 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:32.968712 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea33270-e544-4fb4-8da2-86b027623d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:32.969421 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea33270-e544-4fb4-8da2-86b027623d78 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:32.969814 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea33270-e544-4fb4-8da2-86b027623d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:32.970229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea33270-e544-4fb4-8da2-86b027623d78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:32.971066 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea33270-e544-4fb4-8da2-86b027623d78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:32.973108 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea33270-e544-4fb4-8da2-86b027623d78 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:33.086223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1217/2412] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:15:32 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:15:33.090707 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108ccd9-80d5-4968-84ab-7341b8f4078e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:33.107389 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108ccd9-80d5-4968-84ab-7341b8f4078e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:33.164885 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96712d12-f51d-4907-92d5-a911d24e7338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:33.165529 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96712d12-f51d-4907-92d5-a911d24e7338 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:33.165816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96712d12-f51d-4907-92d5-a911d24e7338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:33.166151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96712d12-f51d-4907-92d5-a911d24e7338 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:33.166790 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96712d12-f51d-4907-92d5-a911d24e7338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:33.168759 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96712d12-f51d-4907-92d5-a911d24e7338 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:33.284256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1196/2413] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:15:33 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:15:33.288601 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea33270-e544-4fb4-8da2-86b027623d78 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:33.299615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea33270-e544-4fb4-8da2-86b027623d78 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:33.304280 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34f321-2a65-4c69-aa53-b76bc014df4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:33.304901 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34f321-2a65-4c69-aa53-b76bc014df4e admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:33.305224 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34f321-2a65-4c69-aa53-b76bc014df4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:33.305625 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34f321-2a65-4c69-aa53-b76bc014df4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:33.306205 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34f321-2a65-4c69-aa53-b76bc014df4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:33.307785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34f321-2a65-4c69-aa53-b76bc014df4e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:33.319632 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1218/2414] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:15:33.324914 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96712d12-f51d-4907-92d5-a911d24e7338 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:33.337290 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96712d12-f51d-4907-92d5-a911d24e7338 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:33.342452 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64c33f-5832-4e2d-a3a6-9064c4485c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:33.343073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64c33f-5832-4e2d-a3a6-9064c4485c76 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:33.343522 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64c33f-5832-4e2d-a3a6-9064c4485c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:33.344031 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64c33f-5832-4e2d-a3a6-9064c4485c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:33.344526 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64c33f-5832-4e2d-a3a6-9064c4485c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:33.345950 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64c33f-5832-4e2d-a3a6-9064c4485c76 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:33.357384 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1197/2415] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:15:33.365223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34f321-2a65-4c69-aa53-b76bc014df4e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:33.376224 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34f321-2a65-4c69-aa53-b76bc014df4e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:33.381056 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ed9b4-e093-44be-8050-407933a0b398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:33.381736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ed9b4-e093-44be-8050-407933a0b398 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:33.382060 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ed9b4-e093-44be-8050-407933a0b398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:33.382346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ed9b4-e093-44be-8050-407933a0b398 admin admin] PATH_INFO: `/v3/projects/126afe8b9ded4b58965e64692d083b05/users/a86c77f42ba04dca9cc1b39a3733d207/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:33.400046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ed9b4-e093-44be-8050-407933a0b398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=126afe8b9ded4b58965e64692d083b05, user_id=a86c77f42ba04dca9cc1b39a3733d207, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:33.402765 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ed9b4-e093-44be-8050-407933a0b398 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:33.467259 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1219/2416] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:15:33 2022] PUT /identity/v3/projects/126afe8b9ded4b58965e64692d083b05/users/a86c77f42ba04dca9cc1b39a3733d207/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:15:33.478682 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e56d06-d5ce-407f-9ea5-2acb10d448be None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:33.478973 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e56d06-d5ce-407f-9ea5-2acb10d448be None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:33.479255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e56d06-d5ce-407f-9ea5-2acb10d448be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:33.508400 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-47e56d06-d5ce-407f-9ea5-2acb10d448be None None] MFA Rules not processed for user `a86c77f42ba04dca9cc1b39a3733d207`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:15:33.509622 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-47e56d06-d5ce-407f-9ea5-2acb10d448be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:33.575925 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1198/2417] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:15:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:15:33.581265 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ed9b4-e093-44be-8050-407933a0b398 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:33.604741 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:15:33.648162 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ed9b4-e093-44be-8050-407933a0b398 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:33.653119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e4953-b571-4c56-9f3a-07e42e91a31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:33.653736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e4953-b571-4c56-9f3a-07e42e91a31d admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:33.654060 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e4953-b571-4c56-9f3a-07e42e91a31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:33.654341 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e4953-b571-4c56-9f3a-07e42e91a31d admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:33.654992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e4953-b571-4c56-9f3a-07e42e91a31d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:33.656935 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e4953-b571-4c56-9f3a-07e42e91a31d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:33.753632 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1220/2418] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:15:33 2022] POST /identity/v3/projects => generated 371 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:15:33.757926 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e56d06-d5ce-407f-9ea5-2acb10d448be None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:33.769005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e56d06-d5ce-407f-9ea5-2acb10d448be None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:33.782558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:15:33.828098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7b103-282a-4d2b-a419-e5307edaab03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:33.828656 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7b103-282a-4d2b-a419-e5307edaab03 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:33.828935 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7b103-282a-4d2b-a419-e5307edaab03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:33.829209 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7b103-282a-4d2b-a419-e5307edaab03 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:33.829873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7b103-282a-4d2b-a419-e5307edaab03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:33.831921 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7b103-282a-4d2b-a419-e5307edaab03 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:33.953262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1199/2419] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:15:33 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:15:33.957420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e4953-b571-4c56-9f3a-07e42e91a31d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:33.968519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e4953-b571-4c56-9f3a-07e42e91a31d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:33.973769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3477fa7-aa88-4a01-adfe-a2b97dfd69bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:33.974401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3477fa7-aa88-4a01-adfe-a2b97dfd69bc admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:33.974686 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3477fa7-aa88-4a01-adfe-a2b97dfd69bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:33.974965 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3477fa7-aa88-4a01-adfe-a2b97dfd69bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:33.975608 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3477fa7-aa88-4a01-adfe-a2b97dfd69bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:33.977219 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3477fa7-aa88-4a01-adfe-a2b97dfd69bc admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:33.988969 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1221/2420] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:15:33.993246 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7b103-282a-4d2b-a419-e5307edaab03 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:34.003361 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7b103-282a-4d2b-a419-e5307edaab03 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:34.008300 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336c132-ddd4-4d73-8416-02e5ab40f581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:34.008959 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336c132-ddd4-4d73-8416-02e5ab40f581 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:34.009241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336c132-ddd4-4d73-8416-02e5ab40f581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:34.009518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336c132-ddd4-4d73-8416-02e5ab40f581 admin admin] PATH_INFO: `/v3/projects/6e0c71878a3b4cf4b3c3cd31309e2ba2/users/7b0ab78d4f6045789938a86ff8708c72/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:34.026353 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336c132-ddd4-4d73-8416-02e5ab40f581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e0c71878a3b4cf4b3c3cd31309e2ba2, user_id=7b0ab78d4f6045789938a86ff8708c72, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:34.029011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336c132-ddd4-4d73-8416-02e5ab40f581 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:34.095315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1200/2421] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:15:33 2022] PUT /identity/v3/projects/6e0c71878a3b4cf4b3c3cd31309e2ba2/users/7b0ab78d4f6045789938a86ff8708c72/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:15:34.099324 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3477fa7-aa88-4a01-adfe-a2b97dfd69bc admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:34.109269 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3477fa7-aa88-4a01-adfe-a2b97dfd69bc admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:34.165051 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae956383-e3c9-4b1f-9f1b-d3d541924e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:34.165659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae956383-e3c9-4b1f-9f1b-d3d541924e1c admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:34.166044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae956383-e3c9-4b1f-9f1b-d3d541924e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:34.166327 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae956383-e3c9-4b1f-9f1b-d3d541924e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:34.166967 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae956383-e3c9-4b1f-9f1b-d3d541924e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:34.168561 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae956383-e3c9-4b1f-9f1b-d3d541924e1c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:34.178855 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1222/2422] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:15:34.183074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336c132-ddd4-4d73-8416-02e5ab40f581 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:34.193215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336c132-ddd4-4d73-8416-02e5ab40f581 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:34.197752 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e9825-cfc1-4fc6-9919-f88d7c16fe4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:34.198539 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e9825-cfc1-4fc6-9919-f88d7c16fe4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:34.198738 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e9825-cfc1-4fc6-9919-f88d7c16fe4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:34.199000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e9825-cfc1-4fc6-9919-f88d7c16fe4c admin admin] PATH_INFO: `/v3/projects/6e0c71878a3b4cf4b3c3cd31309e2ba2/users/7b0ab78d4f6045789938a86ff8708c72/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:34.200987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e9825-cfc1-4fc6-9919-f88d7c16fe4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e0c71878a3b4cf4b3c3cd31309e2ba2, user_id=7b0ab78d4f6045789938a86ff8708c72, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:34.203664 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e9825-cfc1-4fc6-9919-f88d7c16fe4c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:34.213748 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1201/2423] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:15:34 2022] PUT /identity/v3/projects/6e0c71878a3b4cf4b3c3cd31309e2ba2/users/7b0ab78d4f6045789938a86ff8708c72/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:15:34.224359 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2528-b04d-4b75-a4d9-25a14ab9af34 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:34.224655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2528-b04d-4b75-a4d9-25a14ab9af34 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:34.224970 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2528-b04d-4b75-a4d9-25a14ab9af34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:34.253240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3bfa2528-b04d-4b75-a4d9-25a14ab9af34 None None] MFA Rules not processed for user `7b0ab78d4f6045789938a86ff8708c72`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:15:34.254491 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3bfa2528-b04d-4b75-a4d9-25a14ab9af34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:34.321879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1223/2424] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:15:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:15:34.333455 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e9825-cfc1-4fc6-9919-f88d7c16fe4c admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:34.335181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cd2e9825-cfc1-4fc6-9919-f88d7c16fe4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:34.391644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e9825-cfc1-4fc6-9919-f88d7c16fe4c admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:34.395185 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e763f91-036f-4c0b-997b-7bdc14e65b0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:34.395784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e763f91-036f-4c0b-997b-7bdc14e65b0c service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:34.396061 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e763f91-036f-4c0b-997b-7bdc14e65b0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:34.396333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e763f91-036f-4c0b-997b-7bdc14e65b0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:34.405375 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e763f91-036f-4c0b-997b-7bdc14e65b0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:34.407405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e763f91-036f-4c0b-997b-7bdc14e65b0c service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:34.416631 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1202/2425] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:15:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:15:34.629227 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa2528-b04d-4b75-a4d9-25a14ab9af34 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:34.638882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa2528-b04d-4b75-a4d9-25a14ab9af34 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:34.642953 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0848e7-8772-4105-8002-f4c19800503f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:34.643490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0848e7-8772-4105-8002-f4c19800503f service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:34.643769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0848e7-8772-4105-8002-f4c19800503f service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:34.644039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0848e7-8772-4105-8002-f4c19800503f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:34.655839 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:15:34.700905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0848e7-8772-4105-8002-f4c19800503f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:34.702396 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0848e7-8772-4105-8002-f4c19800503f service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:34.712312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1224/2426] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:15:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:15:44.412679 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:15:44.713571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=21 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:15:54.792422 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e763f91-036f-4c0b-997b-7bdc14e65b0c service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:54.793923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6e763f91-036f-4c0b-997b-7bdc14e65b0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:54.854926 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e763f91-036f-4c0b-997b-7bdc14e65b0c service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:54.858620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6239062-a1d8-45d1-aa10-d28776160969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:54.859224 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6239062-a1d8-45d1-aa10-d28776160969 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:54.859513 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6239062-a1d8-45d1-aa10-d28776160969 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:54.859791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6239062-a1d8-45d1-aa10-d28776160969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:54.861543 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a6239062-a1d8-45d1-aa10-d28776160969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:54.962151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6239062-a1d8-45d1-aa10-d28776160969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:54.963940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6239062-a1d8-45d1-aa10-d28776160969 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:54.980879 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1203/2427] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:15:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:15:56.189921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0848e7-8772-4105-8002-f4c19800503f service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:15:56.200897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0848e7-8772-4105-8002-f4c19800503f service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:15:56.205196 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931db3ca-b2d0-43ed-bb74-80ecfa77577d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:15:56.205806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931db3ca-b2d0-43ed-bb74-80ecfa77577d service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:15:56.206135 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931db3ca-b2d0-43ed-bb74-80ecfa77577d service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:15:56.206448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931db3ca-b2d0-43ed-bb74-80ecfa77577d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:15:56.208512 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-931db3ca-b2d0-43ed-bb74-80ecfa77577d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:15:56.223322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931db3ca-b2d0-43ed-bb74-80ecfa77577d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:15:56.224830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931db3ca-b2d0-43ed-bb74-80ecfa77577d service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:15:56.237465 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1225/2428] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:15:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:16:04.974134 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=54 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:06.231666 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=6 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:08.102239 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6239062-a1d8-45d1-aa10-d28776160969 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:08.103786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a6239062-a1d8-45d1-aa10-d28776160969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:08.118281 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6239062-a1d8-45d1-aa10-d28776160969 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:08.123847 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe169ef-6a93-401d-9160-093b377fe642 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:08.124464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe169ef-6a93-401d-9160-093b377fe642 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:08.124749 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe169ef-6a93-401d-9160-093b377fe642 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:08.125031 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe169ef-6a93-401d-9160-093b377fe642 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:08.127685 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8fe169ef-6a93-401d-9160-093b377fe642 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:08.223466 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe169ef-6a93-401d-9160-093b377fe642 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:08.225117 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe169ef-6a93-401d-9160-093b377fe642 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:08.234798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1204/2429] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:16:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:16:17.094687 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c67ae-1fe1-499f-b8f1-615107b1cb5e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:17.095839 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c67ae-1fe1-499f-b8f1-615107b1cb5e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:17.095839 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c67ae-1fe1-499f-b8f1-615107b1cb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:17.124423 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-761c67ae-1fe1-499f-b8f1-615107b1cb5e None None] MFA Rules not processed for user `d76a164ed7e14d9eb2db016b68a84d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:16:17.125760 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-761c67ae-1fe1-499f-b8f1-615107b1cb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:17.184244 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1226/2430] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 14 12:16:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:16:17.327906 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe169ef-6a93-401d-9160-093b377fe642 service nova] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:17.329434 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8fe169ef-6a93-401d-9160-093b377fe642 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:17.412320 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe169ef-6a93-401d-9160-093b377fe642 service nova] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:17.419476 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb98fc-a4e6-4184-8576-fc68af6190cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1047c7d18c1466d989cd895117cd807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:17.420313 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb98fc-a4e6-4184-8576-fc68af6190cf service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:17.420706 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb98fc-a4e6-4184-8576-fc68af6190cf service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:17.421098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb98fc-a4e6-4184-8576-fc68af6190cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:17.433178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb98fc-a4e6-4184-8576-fc68af6190cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:17.434822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb98fc-a4e6-4184-8576-fc68af6190cf service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:17.445013 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1205/2431] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 14 12:16:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 12:16:27.178456 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=29 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:27.440825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=69 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:36.630026 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c67ae-1fe1-499f-b8f1-615107b1cb5e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:36.701097 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c67ae-1fe1-499f-b8f1-615107b1cb5e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:36.705830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18a30c-b056-4e02-bc87-921d3bd0f38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:36.706552 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18a30c-b056-4e02-bc87-921d3bd0f38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:36.706963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18a30c-b056-4e02-bc87-921d3bd0f38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:36.707297 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18a30c-b056-4e02-bc87-921d3bd0f38f admin admin] PATH_INFO: `/v3/users/a86c77f42ba04dca9cc1b39a3733d207` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:36.708433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18a30c-b056-4e02-bc87-921d3bd0f38f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a86c77f42ba04dca9cc1b39a3733d207)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:36.710602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18a30c-b056-4e02-bc87-921d3bd0f38f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:36.943941 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-aa18a30c-b056-4e02-bc87-921d3bd0f38f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a86c77f42ba04dca9cc1b39a3733d207'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:36.957799 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-aa18a30c-b056-4e02-bc87-921d3bd0f38f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a86c77f42ba04dca9cc1b39a3733d207'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:36.965019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-aa18a30c-b056-4e02-bc87-921d3bd0f38f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a86c77f42ba04dca9cc1b39a3733d207'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:37.037717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-aa18a30c-b056-4e02-bc87-921d3bd0f38f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a86c77f42ba04dca9cc1b39a3733d207'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:37.038399 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-aa18a30c-b056-4e02-bc87-921d3bd0f38f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a86c77f42ba04dca9cc1b39a3733d207'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:37.044896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1227/2432] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:16:36 2022] DELETE /identity/v3/users/a86c77f42ba04dca9cc1b39a3733d207 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:16:37.049934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60331eea-a379-421e-97d9-39c5740b0c9a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:37.051242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60331eea-a379-421e-97d9-39c5740b0c9a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:37.051242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60331eea-a379-421e-97d9-39c5740b0c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:37.071644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-60331eea-a379-421e-97d9-39c5740b0c9a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:16:37.072889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-60331eea-a379-421e-97d9-39c5740b0c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:37.137021 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1206/2433] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:16:37.147364 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18a30c-b056-4e02-bc87-921d3bd0f38f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:37.149137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-aa18a30c-b056-4e02-bc87-921d3bd0f38f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:37.211682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18a30c-b056-4e02-bc87-921d3bd0f38f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:37.215526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8cec95-3b54-4bba-9c8b-df42484a27f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:37.216163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8cec95-3b54-4bba-9c8b-df42484a27f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:37.216448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8cec95-3b54-4bba-9c8b-df42484a27f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:37.216726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8cec95-3b54-4bba-9c8b-df42484a27f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:37.227072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8cec95-3b54-4bba-9c8b-df42484a27f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:37.228495 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8cec95-3b54-4bba-9c8b-df42484a27f3 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:37.239209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1228/2434] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:16:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:16:37.697827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60331eea-a379-421e-97d9-39c5740b0c9a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:37.699802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-60331eea-a379-421e-97d9-39c5740b0c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:37.712268 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60331eea-a379-421e-97d9-39c5740b0c9a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:37.718548 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3bd7df-fc20-4a59-84bb-01bb109799aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:37.719418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bd7df-fc20-4a59-84bb-01bb109799aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:37.719704 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bd7df-fc20-4a59-84bb-01bb109799aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:37.720015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bd7df-fc20-4a59-84bb-01bb109799aa admin admin] PATH_INFO: `/v3/projects/126afe8b9ded4b58965e64692d083b05` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:37.721357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3bd7df-fc20-4a59-84bb-01bb109799aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=126afe8b9ded4b58965e64692d083b05)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:37.724150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3bd7df-fc20-4a59-84bb-01bb109799aa admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:37.857689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ad3bd7df-fc20-4a59-84bb-01bb109799aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '126afe8b9ded4b58965e64692d083b05', 'request_id': 'req-ad3bd7df-fc20-4a59-84bb-01bb109799aa'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:37.934643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ad3bd7df-fc20-4a59-84bb-01bb109799aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '126afe8b9ded4b58965e64692d083b05', 'request_id': 'req-ad3bd7df-fc20-4a59-84bb-01bb109799aa'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:37.943448 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ad3bd7df-fc20-4a59-84bb-01bb109799aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '126afe8b9ded4b58965e64692d083b05', 'request_id': 'req-ad3bd7df-fc20-4a59-84bb-01bb109799aa'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:37.957158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ad3bd7df-fc20-4a59-84bb-01bb109799aa admin admin] The token cache is being invalidate because project 126afe8b9ded4b58965e64692d083b05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:16:37.957463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ad3bd7df-fc20-4a59-84bb-01bb109799aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:37.958156 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ad3bd7df-fc20-4a59-84bb-01bb109799aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:37.970166 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1207/2435] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:16:37 2022] DELETE /identity/v3/projects/126afe8b9ded4b58965e64692d083b05 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:16:37.970304 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8cec95-3b54-4bba-9c8b-df42484a27f3 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:37.976549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8cec95-3b54-4bba-9c8b-df42484a27f3 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:37.999978 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:38.052880 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa5463-031e-4b6f-b133-8d1d7895db68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:38.053527 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa5463-031e-4b6f-b133-8d1d7895db68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:38.053799 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa5463-031e-4b6f-b133-8d1d7895db68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:38.054124 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa5463-031e-4b6f-b133-8d1d7895db68 admin admin] PATH_INFO: `/v3/users/7b0ab78d4f6045789938a86ff8708c72` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:38.055310 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa5463-031e-4b6f-b133-8d1d7895db68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b0ab78d4f6045789938a86ff8708c72)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:38.057163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa5463-031e-4b6f-b133-8d1d7895db68 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:38.273552 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-83fa5463-031e-4b6f-b133-8d1d7895db68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b0ab78d4f6045789938a86ff8708c72'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:38.285843 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-83fa5463-031e-4b6f-b133-8d1d7895db68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b0ab78d4f6045789938a86ff8708c72'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:38.292795 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-83fa5463-031e-4b6f-b133-8d1d7895db68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b0ab78d4f6045789938a86ff8708c72'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:38.365033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-83fa5463-031e-4b6f-b133-8d1d7895db68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b0ab78d4f6045789938a86ff8708c72'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:38.365933 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-83fa5463-031e-4b6f-b133-8d1d7895db68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b0ab78d4f6045789938a86ff8708c72'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:38.374254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1229/2436] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:16:37 2022] DELETE /identity/v3/users/7b0ab78d4f6045789938a86ff8708c72 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:16:38.868305 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3bd7df-fc20-4a59-84bb-01bb109799aa admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:38.872731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ad3bd7df-fc20-4a59-84bb-01bb109799aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:38.884952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3bd7df-fc20-4a59-84bb-01bb109799aa admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:38.965634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49219596-8ac4-45c7-b655-cf85b0a0f5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:38.966305 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49219596-8ac4-45c7-b655-cf85b0a0f5c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:38.966593 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49219596-8ac4-45c7-b655-cf85b0a0f5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:38.966874 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49219596-8ac4-45c7-b655-cf85b0a0f5c0 admin admin] PATH_INFO: `/v3/projects/6e0c71878a3b4cf4b3c3cd31309e2ba2` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:38.968170 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49219596-8ac4-45c7-b655-cf85b0a0f5c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e0c71878a3b4cf4b3c3cd31309e2ba2)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:38.970131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49219596-8ac4-45c7-b655-cf85b0a0f5c0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:39.062672 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:39.082696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-49219596-8ac4-45c7-b655-cf85b0a0f5c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e0c71878a3b4cf4b3c3cd31309e2ba2', 'request_id': 'req-49219596-8ac4-45c7-b655-cf85b0a0f5c0'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:39.158370 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-49219596-8ac4-45c7-b655-cf85b0a0f5c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e0c71878a3b4cf4b3c3cd31309e2ba2', 'request_id': 'req-49219596-8ac4-45c7-b655-cf85b0a0f5c0'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:39.166036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-49219596-8ac4-45c7-b655-cf85b0a0f5c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e0c71878a3b4cf4b3c3cd31309e2ba2', 'request_id': 'req-49219596-8ac4-45c7-b655-cf85b0a0f5c0'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:39.181396 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-49219596-8ac4-45c7-b655-cf85b0a0f5c0 admin admin] The token cache is being invalidate because project 6e0c71878a3b4cf4b3c3cd31309e2ba2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:16:39.181735 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-49219596-8ac4-45c7-b655-cf85b0a0f5c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:39.182417 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-49219596-8ac4-45c7-b655-cf85b0a0f5c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:39.184193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1208/2437] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:16:38 2022] DELETE /identity/v3/projects/6e0c71878a3b4cf4b3c3cd31309e2ba2 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:16:39.188953 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc6c78-644d-4c61-b82e-02a84851c583 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:39.189695 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc6c78-644d-4c61-b82e-02a84851c583 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:39.189695 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc6c78-644d-4c61-b82e-02a84851c583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:39.214158 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8dbc6c78-644d-4c61-b82e-02a84851c583 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:16:39.215614 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8dbc6c78-644d-4c61-b82e-02a84851c583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:39.258887 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49219596-8ac4-45c7-b655-cf85b0a0f5c0 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:39.260532 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-49219596-8ac4-45c7-b655-cf85b0a0f5c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:39.306052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1230/2438] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:16:39.312979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2321ff3-aef0-47a4-8585-3236821f6438 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:39.313908 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2321ff3-aef0-47a4-8585-3236821f6438 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:39.314073 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2321ff3-aef0-47a4-8585-3236821f6438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:39.347230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49219596-8ac4-45c7-b655-cf85b0a0f5c0 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:39.348952 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a2321ff3-aef0-47a4-8585-3236821f6438 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:16:39.350876 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a2321ff3-aef0-47a4-8585-3236821f6438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:39.353070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3690a-95c8-4efe-895a-f60bc652398d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b05d0e65e9645f3810a6d96c8fed9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:39.354124 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3690a-95c8-4efe-895a-f60bc652398d service glance] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:39.354586 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3690a-95c8-4efe-895a-f60bc652398d service glance] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:39.355031 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3690a-95c8-4efe-895a-f60bc652398d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:39.357674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3ac3690a-95c8-4efe-895a-f60bc652398d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:39.363820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1231/2439] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:16:39.370555 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2321ff3-aef0-47a4-8585-3236821f6438 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:39.386007 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2321ff3-aef0-47a4-8585-3236821f6438 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:39.394581 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf473259-89d0-4bb4-a1ab-59b7d819978f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:39.395553 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf473259-89d0-4bb4-a1ab-59b7d819978f admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:39.395951 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf473259-89d0-4bb4-a1ab-59b7d819978f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:39.396408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf473259-89d0-4bb4-a1ab-59b7d819978f admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:39.397487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf473259-89d0-4bb4-a1ab-59b7d819978f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:16:39.398022 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf473259-89d0-4bb4-a1ab-59b7d819978f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:39.401197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf473259-89d0-4bb4-a1ab-59b7d819978f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:39.419438 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1232/2440] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 12:16:39.424038 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf473259-89d0-4bb4-a1ab-59b7d819978f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:39.437486 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf473259-89d0-4bb4-a1ab-59b7d819978f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:39.443616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdfc1c-1f60-4cea-9556-75d39791f0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:39.444622 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdfc1c-1f60-4cea-9556-75d39791f0c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:39.444864 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdfc1c-1f60-4cea-9556-75d39791f0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:39.445269 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdfc1c-1f60-4cea-9556-75d39791f0c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:39.446175 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbdfc1c-1f60-4cea-9556-75d39791f0c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:39.448075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbdfc1c-1f60-4cea-9556-75d39791f0c3 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:39.458916 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3690a-95c8-4efe-895a-f60bc652398d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:39.460371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3690a-95c8-4efe-895a-f60bc652398d service glance] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:39.471591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1209/2441] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:16:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:16:39.578280 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:39.589318 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1233/2442] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:16:39 2022] POST /identity/v3/projects => generated 367 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:16:39.590561 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3690a-95c8-4efe-895a-f60bc652398d service glance] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:39.603221 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3690a-95c8-4efe-895a-f60bc652398d service glance] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:39.664302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:39.671946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196287a8-2405-4760-bb12-a71ab94b132e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:39.672585 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196287a8-2405-4760-bb12-a71ab94b132e admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:39.672884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196287a8-2405-4760-bb12-a71ab94b132e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:39.673164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196287a8-2405-4760-bb12-a71ab94b132e admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:39.673831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196287a8-2405-4760-bb12-a71ab94b132e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:39.676001 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196287a8-2405-4760-bb12-a71ab94b132e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:39.729017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdfc1c-1f60-4cea-9556-75d39791f0c3 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:39.730986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6cbdfc1c-1f60-4cea-9556-75d39791f0c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:39.798118 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdfc1c-1f60-4cea-9556-75d39791f0c3 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:39.802749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095d26e-d41f-4796-815b-16e20a029e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:39.803370 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095d26e-d41f-4796-815b-16e20a029e8f service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:39.803651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095d26e-d41f-4796-815b-16e20a029e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:39.803955 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095d26e-d41f-4796-815b-16e20a029e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:39.805964 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6095d26e-d41f-4796-815b-16e20a029e8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:39.816695 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095d26e-d41f-4796-815b-16e20a029e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:39.820970 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095d26e-d41f-4796-815b-16e20a029e8f service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:39.830062 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1234/2443] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:16:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:16:39.855962 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1210/2444] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:16:39 2022] POST /identity/v3/users => generated 284 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:16:39.860674 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095d26e-d41f-4796-815b-16e20a029e8f service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:39.878115 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095d26e-d41f-4796-815b-16e20a029e8f service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:39.883707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369460cb-e59c-4fa5-9a25-25f8fbd1ed29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:39.884340 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369460cb-e59c-4fa5-9a25-25f8fbd1ed29 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:39.884626 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369460cb-e59c-4fa5-9a25-25f8fbd1ed29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:39.884909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369460cb-e59c-4fa5-9a25-25f8fbd1ed29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:39.885569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369460cb-e59c-4fa5-9a25-25f8fbd1ed29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:39.887480 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369460cb-e59c-4fa5-9a25-25f8fbd1ed29 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:39.900462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1235/2445] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:16:39.911726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196287a8-2405-4760-bb12-a71ab94b132e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:39.919508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369460cb-e59c-4fa5-9a25-25f8fbd1ed29 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:39.923278 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196287a8-2405-4760-bb12-a71ab94b132e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:39.930216 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369460cb-e59c-4fa5-9a25-25f8fbd1ed29 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:39.931170 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c889805-54ac-4587-8d83-0b4ff3a01e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:39.932335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c889805-54ac-4587-8d83-0b4ff3a01e42 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:39.932803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c889805-54ac-4587-8d83-0b4ff3a01e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:39.933202 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c889805-54ac-4587-8d83-0b4ff3a01e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:39.934177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cb2dd-5bad-4a4c-acb3-09e56f4ef196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:39.934317 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c889805-54ac-4587-8d83-0b4ff3a01e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:39.934914 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cb2dd-5bad-4a4c-acb3-09e56f4ef196 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:39.940028 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cb2dd-5bad-4a4c-acb3-09e56f4ef196 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:39.940028 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cb2dd-5bad-4a4c-acb3-09e56f4ef196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:39.940264 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c889805-54ac-4587-8d83-0b4ff3a01e42 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:39.940372 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ae8cb2dd-5bad-4a4c-acb3-09e56f4ef196 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:39.956011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1211/2446] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:16:39.962010 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c889805-54ac-4587-8d83-0b4ff3a01e42 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:39.973369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c889805-54ac-4587-8d83-0b4ff3a01e42 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:39.980484 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213e2d5-a2a8-4c86-a844-c4c186bfb38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:39.981622 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213e2d5-a2a8-4c86-a844-c4c186bfb38d admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:39.981995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213e2d5-a2a8-4c86-a844-c4c186bfb38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:39.982346 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213e2d5-a2a8-4c86-a844-c4c186bfb38d admin admin] PATH_INFO: `/v3/projects/0b5a1dd24d21483c9a492b62f9bcd432/users/d9145ea9a9464bd5af1c2325a33bb55d/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:40.000551 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213e2d5-a2a8-4c86-a844-c4c186bfb38d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b5a1dd24d21483c9a492b62f9bcd432, user_id=d9145ea9a9464bd5af1c2325a33bb55d, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:40.004037 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213e2d5-a2a8-4c86-a844-c4c186bfb38d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:40.028998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8cb2dd-5bad-4a4c-acb3-09e56f4ef196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:40.028998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8cb2dd-5bad-4a4c-acb3-09e56f4ef196 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:40.044487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1236/2447] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:16:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:16:40.113636 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1212/2448] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:16:39 2022] PUT /identity/v3/projects/0b5a1dd24d21483c9a492b62f9bcd432/users/d9145ea9a9464bd5af1c2325a33bb55d/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:16:40.123614 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cb2dd-5bad-4a4c-acb3-09e56f4ef196 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:40.126399 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ae8cb2dd-5bad-4a4c-acb3-09e56f4ef196 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:40.126561 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b193c1f-fd59-4f94-aa92-fe50ff6b4104 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:40.126667 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b193c1f-fd59-4f94-aa92-fe50ff6b4104 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:40.128682 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b193c1f-fd59-4f94-aa92-fe50ff6b4104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:40.165573 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-5b193c1f-fd59-4f94-aa92-fe50ff6b4104 None None] MFA Rules not processed for user `d9145ea9a9464bd5af1c2325a33bb55d`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:16:40.166979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5b193c1f-fd59-4f94-aa92-fe50ff6b4104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:40.225974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cb2dd-5bad-4a4c-acb3-09e56f4ef196 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:40.248963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:40.259983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1213/2449] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:16:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:16:40.264487 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b193c1f-fd59-4f94-aa92-fe50ff6b4104 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:40.308356 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473cbf5-aab4-48ba-8d1f-61bb543221b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca10dd7ed714edb8b87017deecc41b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:40.309319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473cbf5-aab4-48ba-8d1f-61bb543221b9 service placement] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:40.309790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473cbf5-aab4-48ba-8d1f-61bb543221b9 service placement] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:40.310419 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473cbf5-aab4-48ba-8d1f-61bb543221b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:40.313048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b473cbf5-aab4-48ba-8d1f-61bb543221b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:40.327241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473cbf5-aab4-48ba-8d1f-61bb543221b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:40.328900 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473cbf5-aab4-48ba-8d1f-61bb543221b9 service placement] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:40.340606 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1237/2450] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:16:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:16:40.344698 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b193c1f-fd59-4f94-aa92-fe50ff6b4104 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:40.353501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c104e-7468-4e04-b381-e5a8920e33c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:40.354538 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c104e-7468-4e04-b381-e5a8920e33c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:40.354911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c104e-7468-4e04-b381-e5a8920e33c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:40.355238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c104e-7468-4e04-b381-e5a8920e33c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:40.356262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c104e-7468-4e04-b381-e5a8920e33c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:40.359950 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c104e-7468-4e04-b381-e5a8920e33c1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:40.474035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1214/2451] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:16:40 2022] POST /identity/v3/projects => generated 369 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:16:40.477816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473cbf5-aab4-48ba-8d1f-61bb543221b9 service placement] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:40.489639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473cbf5-aab4-48ba-8d1f-61bb543221b9 service placement] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:40.568349 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dcfc2a-0e57-4344-afb3-29113ee2e05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:40.568995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcfc2a-0e57-4344-afb3-29113ee2e05c admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:40.569275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcfc2a-0e57-4344-afb3-29113ee2e05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:40.569551 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcfc2a-0e57-4344-afb3-29113ee2e05c admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:40.570232 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dcfc2a-0e57-4344-afb3-29113ee2e05c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:40.572176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dcfc2a-0e57-4344-afb3-29113ee2e05c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:40.718798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1238/2452] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:16:40 2022] POST /identity/v3/users => generated 291 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:16:40.723277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c104e-7468-4e04-b381-e5a8920e33c1 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:40.733848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c104e-7468-4e04-b381-e5a8920e33c1 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:40.739828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301772f5-8926-47cb-ba76-4dcaf6c7ce45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:40.740439 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301772f5-8926-47cb-ba76-4dcaf6c7ce45 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:40.740939 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301772f5-8926-47cb-ba76-4dcaf6c7ce45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:40.741399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301772f5-8926-47cb-ba76-4dcaf6c7ce45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:40.742083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301772f5-8926-47cb-ba76-4dcaf6c7ce45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:40.744035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301772f5-8926-47cb-ba76-4dcaf6c7ce45 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:40.756392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1215/2453] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:16:40.763310 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dcfc2a-0e57-4344-afb3-29113ee2e05c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:40.779101 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dcfc2a-0e57-4344-afb3-29113ee2e05c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:40.786045 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d34ae2-d9c6-4460-9211-269e8a58af55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:40.786045 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d34ae2-d9c6-4460-9211-269e8a58af55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:40.786045 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d34ae2-d9c6-4460-9211-269e8a58af55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:40.786858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d34ae2-d9c6-4460-9211-269e8a58af55 admin admin] PATH_INFO: `/v3/projects/7083cc16ca3f43b1b864629b690de4bb/users/c9ea80acdd8546c983f7f3243536306c/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:40.804359 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d34ae2-d9c6-4460-9211-269e8a58af55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7083cc16ca3f43b1b864629b690de4bb, user_id=c9ea80acdd8546c983f7f3243536306c, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:40.807283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d34ae2-d9c6-4460-9211-269e8a58af55 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:40.924776 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1239/2454] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:16:40 2022] PUT /identity/v3/projects/7083cc16ca3f43b1b864629b690de4bb/users/c9ea80acdd8546c983f7f3243536306c/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:16:40.929359 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301772f5-8926-47cb-ba76-4dcaf6c7ce45 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:40.942745 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301772f5-8926-47cb-ba76-4dcaf6c7ce45 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:40.953115 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:41.006831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef7b11-2af7-47b9-890b-2a8b22289cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:41.007534 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef7b11-2af7-47b9-890b-2a8b22289cec admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:41.007863 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef7b11-2af7-47b9-890b-2a8b22289cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:41.008206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef7b11-2af7-47b9-890b-2a8b22289cec admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:41.008966 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef7b11-2af7-47b9-890b-2a8b22289cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:41.011331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef7b11-2af7-47b9-890b-2a8b22289cec admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:41.024313 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1216/2455] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:16:41.027535 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d34ae2-d9c6-4460-9211-269e8a58af55 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:41.039463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d34ae2-d9c6-4460-9211-269e8a58af55 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:41.045236 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a7468-babe-404f-b222-5b7179c3edc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:41.046310 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a7468-babe-404f-b222-5b7179c3edc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:41.046595 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a7468-babe-404f-b222-5b7179c3edc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:41.046874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a7468-babe-404f-b222-5b7179c3edc2 admin admin] PATH_INFO: `/v3/projects/7083cc16ca3f43b1b864629b690de4bb/users/c9ea80acdd8546c983f7f3243536306c/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:41.049057 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a7468-babe-404f-b222-5b7179c3edc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7083cc16ca3f43b1b864629b690de4bb, user_id=c9ea80acdd8546c983f7f3243536306c, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:41.052321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a7468-babe-404f-b222-5b7179c3edc2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:41.064586 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1240/2456] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:16:41 2022] PUT /identity/v3/projects/7083cc16ca3f43b1b864629b690de4bb/users/c9ea80acdd8546c983f7f3243536306c/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:16:41.074996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5804993-dfa2-498b-a401-a115484677cb None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:41.075305 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5804993-dfa2-498b-a401-a115484677cb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:41.075615 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5804993-dfa2-498b-a401-a115484677cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:41.106783 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b5804993-dfa2-498b-a401-a115484677cb None None] MFA Rules not processed for user `c9ea80acdd8546c983f7f3243536306c`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:16:41.108459 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b5804993-dfa2-498b-a401-a115484677cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:41.195158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1217/2457] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:16:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:16:41.207929 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a7468-babe-404f-b222-5b7179c3edc2 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:41.209693 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f77a7468-babe-404f-b222-5b7179c3edc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:41.272091 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a7468-babe-404f-b222-5b7179c3edc2 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:41.276166 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07cd73-9f96-4c57-b3ea-5ac509dc0c91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:41.276770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07cd73-9f96-4c57-b3ea-5ac509dc0c91 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:41.277157 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07cd73-9f96-4c57-b3ea-5ac509dc0c91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:41.277434 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07cd73-9f96-4c57-b3ea-5ac509dc0c91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:41.287493 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07cd73-9f96-4c57-b3ea-5ac509dc0c91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:41.288909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07cd73-9f96-4c57-b3ea-5ac509dc0c91 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:41.299649 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1241/2458] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:16:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:16:41.419858 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5804993-dfa2-498b-a401-a115484677cb None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:41.422144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b5804993-dfa2-498b-a401-a115484677cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:41.435589 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5804993-dfa2-498b-a401-a115484677cb None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:41.439404 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f72fa-32a7-43fd-af12-61cbbbd9f318 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:41.440673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f72fa-32a7-43fd-af12-61cbbbd9f318 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:41.440673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f72fa-32a7-43fd-af12-61cbbbd9f318 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:41.440673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f72fa-32a7-43fd-af12-61cbbbd9f318 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:41.511017 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f72fa-32a7-43fd-af12-61cbbbd9f318 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:41.512660 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f72fa-32a7-43fd-af12-61cbbbd9f318 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:41.523315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1218/2459] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:16:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:16:42.995982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5127b47-bc0a-415c-b8ad-d2538cd825e2 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:42.996546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5127b47-bc0a-415c-b8ad-d2538cd825e2 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:42.996546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5127b47-bc0a-415c-b8ad-d2538cd825e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:43.005126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:43.047998 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e5127b47-bc0a-415c-b8ad-d2538cd825e2 None None] MFA Rules not processed for user `d76a164ed7e14d9eb2db016b68a84d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:16:43.049127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e5127b47-bc0a-415c-b8ad-d2538cd825e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:43.116702 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1242/2460] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 14 12:16:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:16:43.129240 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f72fa-32a7-43fd-af12-61cbbbd9f318 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:43.130664 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7c2f72fa-32a7-43fd-af12-61cbbbd9f318 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:43.168546 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:43.220807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f72fa-32a7-43fd-af12-61cbbbd9f318 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:43.225985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becfaf9-9dc9-4900-8efc-50a62d0e3585 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1047c7d18c1466d989cd895117cd807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:43.226588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becfaf9-9dc9-4900-8efc-50a62d0e3585 service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:43.226865 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becfaf9-9dc9-4900-8efc-50a62d0e3585 service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:43.227140 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becfaf9-9dc9-4900-8efc-50a62d0e3585 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:43.235221 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7becfaf9-9dc9-4900-8efc-50a62d0e3585 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:43.236808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7becfaf9-9dc9-4900-8efc-50a62d0e3585 service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:43.245236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1219/2461] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 14 12:16:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 12:16:46.733540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5127b47-bc0a-415c-b8ad-d2538cd825e2 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:46.802536 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5127b47-bc0a-415c-b8ad-d2538cd825e2 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:46.807537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474c693-a7a3-4cf9-b91f-0d9e4df441c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:46.809003 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474c693-a7a3-4cf9-b91f-0d9e4df441c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:46.809333 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474c693-a7a3-4cf9-b91f-0d9e4df441c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:46.809685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474c693-a7a3-4cf9-b91f-0d9e4df441c6 admin admin] PATH_INFO: `/v3/users/d9145ea9a9464bd5af1c2325a33bb55d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:46.811015 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474c693-a7a3-4cf9-b91f-0d9e4df441c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9145ea9a9464bd5af1c2325a33bb55d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:46.813079 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474c693-a7a3-4cf9-b91f-0d9e4df441c6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:47.036682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3474c693-a7a3-4cf9-b91f-0d9e4df441c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9145ea9a9464bd5af1c2325a33bb55d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:47.050501 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3474c693-a7a3-4cf9-b91f-0d9e4df441c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9145ea9a9464bd5af1c2325a33bb55d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:47.059598 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3474c693-a7a3-4cf9-b91f-0d9e4df441c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9145ea9a9464bd5af1c2325a33bb55d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:47.063414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:47.129106 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3474c693-a7a3-4cf9-b91f-0d9e4df441c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9145ea9a9464bd5af1c2325a33bb55d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:47.129797 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3474c693-a7a3-4cf9-b91f-0d9e4df441c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9145ea9a9464bd5af1c2325a33bb55d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:47.137062 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1243/2462] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:16:46 2022] DELETE /identity/v3/users/d9145ea9a9464bd5af1c2325a33bb55d => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:16:47.142052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761bed-cc8e-4e18-9c69-f4ffb5a585df None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:47.142578 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761bed-cc8e-4e18-9c69-f4ffb5a585df None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:47.143660 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761bed-cc8e-4e18-9c69-f4ffb5a585df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:47.166223 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-2a761bed-cc8e-4e18-9c69-f4ffb5a585df None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:16:47.167758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2a761bed-cc8e-4e18-9c69-f4ffb5a585df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:47.230279 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1220/2463] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:16:47.240455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474c693-a7a3-4cf9-b91f-0d9e4df441c6 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:47.242198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3474c693-a7a3-4cf9-b91f-0d9e4df441c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:47.306052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474c693-a7a3-4cf9-b91f-0d9e4df441c6 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:47.309735 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d5f88-d83f-408d-b218-6245889dee5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:47.310389 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d5f88-d83f-408d-b218-6245889dee5d service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:47.310668 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d5f88-d83f-408d-b218-6245889dee5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:47.311085 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d5f88-d83f-408d-b218-6245889dee5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:47.322126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d5f88-d83f-408d-b218-6245889dee5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:47.323664 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d5f88-d83f-408d-b218-6245889dee5d service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:47.335160 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1244/2464] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:16:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:16:47.808925 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a761bed-cc8e-4e18-9c69-f4ffb5a585df None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:47.810539 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2a761bed-cc8e-4e18-9c69-f4ffb5a585df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:47.825203 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a761bed-cc8e-4e18-9c69-f4ffb5a585df None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:47.849241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3bb59-4795-411d-9fb7-db44257fbc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:47.852959 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3bb59-4795-411d-9fb7-db44257fbc25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:47.852959 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3bb59-4795-411d-9fb7-db44257fbc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:47.853377 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3bb59-4795-411d-9fb7-db44257fbc25 admin admin] PATH_INFO: `/v3/projects/0b5a1dd24d21483c9a492b62f9bcd432` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:47.856686 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3bb59-4795-411d-9fb7-db44257fbc25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b5a1dd24d21483c9a492b62f9bcd432)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:47.863753 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3bb59-4795-411d-9fb7-db44257fbc25 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:48.068531 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-67b3bb59-4795-411d-9fb7-db44257fbc25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b5a1dd24d21483c9a492b62f9bcd432', 'request_id': 'req-67b3bb59-4795-411d-9fb7-db44257fbc25'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:48.192110 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-67b3bb59-4795-411d-9fb7-db44257fbc25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b5a1dd24d21483c9a492b62f9bcd432', 'request_id': 'req-67b3bb59-4795-411d-9fb7-db44257fbc25'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:48.200458 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-67b3bb59-4795-411d-9fb7-db44257fbc25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b5a1dd24d21483c9a492b62f9bcd432', 'request_id': 'req-67b3bb59-4795-411d-9fb7-db44257fbc25'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:48.213269 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-67b3bb59-4795-411d-9fb7-db44257fbc25 admin admin] The token cache is being invalidate because project 0b5a1dd24d21483c9a492b62f9bcd432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:16:48.213710 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-67b3bb59-4795-411d-9fb7-db44257fbc25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:48.214239 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-67b3bb59-4795-411d-9fb7-db44257fbc25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:48.215865 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1221/2465] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:16:47 2022] DELETE /identity/v3/projects/0b5a1dd24d21483c9a492b62f9bcd432 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:16:48.220365 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d5f88-d83f-408d-b218-6245889dee5d service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:48.230705 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d5f88-d83f-408d-b218-6245889dee5d service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:48.303433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06629b-6be5-4b43-a73f-18411e77d4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:48.304075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06629b-6be5-4b43-a73f-18411e77d4ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:48.304361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06629b-6be5-4b43-a73f-18411e77d4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:48.304643 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06629b-6be5-4b43-a73f-18411e77d4ff admin admin] PATH_INFO: `/v3/users/c9ea80acdd8546c983f7f3243536306c` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:48.305785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06629b-6be5-4b43-a73f-18411e77d4ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9ea80acdd8546c983f7f3243536306c)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:48.307735 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06629b-6be5-4b43-a73f-18411e77d4ff admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:48.474422 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ae06629b-6be5-4b43-a73f-18411e77d4ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9ea80acdd8546c983f7f3243536306c'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:48.490549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ae06629b-6be5-4b43-a73f-18411e77d4ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9ea80acdd8546c983f7f3243536306c'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:48.499686 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ae06629b-6be5-4b43-a73f-18411e77d4ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9ea80acdd8546c983f7f3243536306c'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:48.590575 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ae06629b-6be5-4b43-a73f-18411e77d4ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9ea80acdd8546c983f7f3243536306c'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:48.591246 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ae06629b-6be5-4b43-a73f-18411e77d4ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9ea80acdd8546c983f7f3243536306c'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:48.598498 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1245/2466] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:16:48 2022] DELETE /identity/v3/users/c9ea80acdd8546c983f7f3243536306c => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:16:49.098325 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3bb59-4795-411d-9fb7-db44257fbc25 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:49.101288 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-67b3bb59-4795-411d-9fb7-db44257fbc25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:49.123944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3bb59-4795-411d-9fb7-db44257fbc25 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:49.156667 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:49.206867 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d7787-dae7-4f58-9022-cf6ceaf6a546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:49.207569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d7787-dae7-4f58-9022-cf6ceaf6a546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:49.207802 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d7787-dae7-4f58-9022-cf6ceaf6a546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:49.208085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d7787-dae7-4f58-9022-cf6ceaf6a546 admin admin] PATH_INFO: `/v3/projects/7083cc16ca3f43b1b864629b690de4bb` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:49.209280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d7787-dae7-4f58-9022-cf6ceaf6a546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7083cc16ca3f43b1b864629b690de4bb)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:49.219183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d7787-dae7-4f58-9022-cf6ceaf6a546 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:49.349129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-069d7787-dae7-4f58-9022-cf6ceaf6a546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7083cc16ca3f43b1b864629b690de4bb', 'request_id': 'req-069d7787-dae7-4f58-9022-cf6ceaf6a546'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:49.419461 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-069d7787-dae7-4f58-9022-cf6ceaf6a546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7083cc16ca3f43b1b864629b690de4bb', 'request_id': 'req-069d7787-dae7-4f58-9022-cf6ceaf6a546'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:49.426640 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-069d7787-dae7-4f58-9022-cf6ceaf6a546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7083cc16ca3f43b1b864629b690de4bb', 'request_id': 'req-069d7787-dae7-4f58-9022-cf6ceaf6a546'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:49.439553 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-069d7787-dae7-4f58-9022-cf6ceaf6a546 admin admin] The token cache is being invalidate because project 7083cc16ca3f43b1b864629b690de4bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:16:49.439811 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-069d7787-dae7-4f58-9022-cf6ceaf6a546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:49.441031 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-069d7787-dae7-4f58-9022-cf6ceaf6a546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:49.442700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1222/2467] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:16:49 2022] DELETE /identity/v3/projects/7083cc16ca3f43b1b864629b690de4bb => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:16:49.448927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d7a6d-3d11-4e81-bacb-25d4a1acff8f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:49.449464 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d7a6d-3d11-4e81-bacb-25d4a1acff8f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:49.449570 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d7a6d-3d11-4e81-bacb-25d4a1acff8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:49.472779 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7b5d7a6d-3d11-4e81-bacb-25d4a1acff8f None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:16:49.474902 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7b5d7a6d-3d11-4e81-bacb-25d4a1acff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:49.490945 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:49.572672 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1246/2468] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:16:49.580730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70b754-03fe-4082-bb82-2b1a54447f3d None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:49.582134 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70b754-03fe-4082-bb82-2b1a54447f3d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:49.582576 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70b754-03fe-4082-bb82-2b1a54447f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:49.611235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-cc70b754-03fe-4082-bb82-2b1a54447f3d None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:16:49.612537 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cc70b754-03fe-4082-bb82-2b1a54447f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:49.623976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1223/2469] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:16:49.628890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d7a6d-3d11-4e81-bacb-25d4a1acff8f None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:49.639549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d7a6d-3d11-4e81-bacb-25d4a1acff8f None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:49.644924 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a77340-0486-486d-95db-bdf97ab01888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:49.645614 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a77340-0486-486d-95db-bdf97ab01888 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:49.646410 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a77340-0486-486d-95db-bdf97ab01888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:49.646410 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a77340-0486-486d-95db-bdf97ab01888 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:49.646868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a77340-0486-486d-95db-bdf97ab01888 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:16:49.647177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a77340-0486-486d-95db-bdf97ab01888 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:49.649189 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a77340-0486-486d-95db-bdf97ab01888 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:49.662641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1247/2470] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:16:49.667326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70b754-03fe-4082-bb82-2b1a54447f3d None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:49.677566 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70b754-03fe-4082-bb82-2b1a54447f3d None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:49.682404 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0559d-569c-4889-9faa-1c60cad43284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:49.683032 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0559d-569c-4889-9faa-1c60cad43284 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:49.683342 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0559d-569c-4889-9faa-1c60cad43284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:49.683620 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0559d-569c-4889-9faa-1c60cad43284 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:49.684262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0559d-569c-4889-9faa-1c60cad43284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:49.686236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0559d-569c-4889-9faa-1c60cad43284 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:49.826568 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1224/2471] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:16:49 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:16:49.827861 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a77340-0486-486d-95db-bdf97ab01888 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:49.840033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a77340-0486-486d-95db-bdf97ab01888 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:49.906431 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883433d-ab1a-4fb8-a98d-be36e0b419e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:49.906955 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883433d-ab1a-4fb8-a98d-be36e0b419e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:49.910023 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883433d-ab1a-4fb8-a98d-be36e0b419e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:49.910023 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883433d-ab1a-4fb8-a98d-be36e0b419e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:49.910023 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883433d-ab1a-4fb8-a98d-be36e0b419e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:49.910586 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883433d-ab1a-4fb8-a98d-be36e0b419e8 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:50.012353 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1248/2472] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:16:49 2022] POST /identity/v3/users => generated 292 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:16:50.016714 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0559d-569c-4889-9faa-1c60cad43284 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:50.029624 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0559d-569c-4889-9faa-1c60cad43284 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:50.040302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf57fca-3a90-4a93-9c8e-ebc150bff60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:50.040302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf57fca-3a90-4a93-9c8e-ebc150bff60a admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:50.041198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf57fca-3a90-4a93-9c8e-ebc150bff60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:50.041198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf57fca-3a90-4a93-9c8e-ebc150bff60a admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:50.041898 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf57fca-3a90-4a93-9c8e-ebc150bff60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:50.043609 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf57fca-3a90-4a93-9c8e-ebc150bff60a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:50.058751 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1225/2473] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:16:50.060622 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883433d-ab1a-4fb8-a98d-be36e0b419e8 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:50.075315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883433d-ab1a-4fb8-a98d-be36e0b419e8 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:50.081426 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b879ede-25ea-4801-8079-2b0b2c22991b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:50.082481 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b879ede-25ea-4801-8079-2b0b2c22991b admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:50.082960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b879ede-25ea-4801-8079-2b0b2c22991b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:50.083426 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b879ede-25ea-4801-8079-2b0b2c22991b admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:50.084521 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b879ede-25ea-4801-8079-2b0b2c22991b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:50.087516 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b879ede-25ea-4801-8079-2b0b2c22991b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:50.105702 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1249/2474] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:16:50.107450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf57fca-3a90-4a93-9c8e-ebc150bff60a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:50.121527 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf57fca-3a90-4a93-9c8e-ebc150bff60a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:50.129604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6eb98-9192-484c-8a1c-8041cc5ee37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:50.130681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6eb98-9192-484c-8a1c-8041cc5ee37a admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:50.131100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6eb98-9192-484c-8a1c-8041cc5ee37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:50.131502 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6eb98-9192-484c-8a1c-8041cc5ee37a admin admin] PATH_INFO: `/v3/projects/b39e710def5f4542b56a21d2b90cdcce/users/2872ce9687574666b79253f356bbc7db/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:50.149274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6eb98-9192-484c-8a1c-8041cc5ee37a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b39e710def5f4542b56a21d2b90cdcce, user_id=2872ce9687574666b79253f356bbc7db, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:50.152151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6eb98-9192-484c-8a1c-8041cc5ee37a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:50.220923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1226/2475] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:16:50 2022] PUT /identity/v3/projects/b39e710def5f4542b56a21d2b90cdcce/users/2872ce9687574666b79253f356bbc7db/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:16:50.233597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96853de8-306a-4142-8a1a-17d580350f4b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:50.234213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96853de8-306a-4142-8a1a-17d580350f4b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:50.234638 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96853de8-306a-4142-8a1a-17d580350f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:50.279032 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-96853de8-306a-4142-8a1a-17d580350f4b None None] MFA Rules not processed for user `2872ce9687574666b79253f356bbc7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:16:50.280278 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-96853de8-306a-4142-8a1a-17d580350f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:50.360801 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1250/2476] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:16:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:16:50.374604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6eb98-9192-484c-8a1c-8041cc5ee37a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:50.376397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-13e6eb98-9192-484c-8a1c-8041cc5ee37a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:50.459800 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6eb98-9192-484c-8a1c-8041cc5ee37a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:50.467545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a47e68-9e72-4d3c-b585-02d3b748ce01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:50.468194 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a47e68-9e72-4d3c-b585-02d3b748ce01 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:50.468640 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a47e68-9e72-4d3c-b585-02d3b748ce01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:50.468919 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a47e68-9e72-4d3c-b585-02d3b748ce01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:50.477297 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:50.480057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a47e68-9e72-4d3c-b585-02d3b748ce01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:50.481753 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a47e68-9e72-4d3c-b585-02d3b748ce01 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:50.494945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1227/2477] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:16:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:16:50.515983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96853de8-306a-4142-8a1a-17d580350f4b None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:50.536858 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:50.619582 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96853de8-306a-4142-8a1a-17d580350f4b None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:50.626284 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a2248-a56e-4587-91a6-4ea2336a3e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:50.633423 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a2248-a56e-4587-91a6-4ea2336a3e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:50.633423 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a2248-a56e-4587-91a6-4ea2336a3e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:50.633423 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a2248-a56e-4587-91a6-4ea2336a3e5a admin admin] PATH_INFO: `/v3/users/2872ce9687574666b79253f356bbc7db` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:50.633423 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a2248-a56e-4587-91a6-4ea2336a3e5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2872ce9687574666b79253f356bbc7db)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:50.633423 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a2248-a56e-4587-91a6-4ea2336a3e5a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:50.864400 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-656a2248-a56e-4587-91a6-4ea2336a3e5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2872ce9687574666b79253f356bbc7db'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:50.884357 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-656a2248-a56e-4587-91a6-4ea2336a3e5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2872ce9687574666b79253f356bbc7db'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:50.894250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-656a2248-a56e-4587-91a6-4ea2336a3e5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2872ce9687574666b79253f356bbc7db'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:50.963712 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-656a2248-a56e-4587-91a6-4ea2336a3e5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2872ce9687574666b79253f356bbc7db'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:50.964353 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-656a2248-a56e-4587-91a6-4ea2336a3e5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2872ce9687574666b79253f356bbc7db'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:50.971689 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1251/2478] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:16:50 2022] DELETE /identity/v3/users/2872ce9687574666b79253f356bbc7db => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:16:50.977031 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e11a48-7b8e-4420-93c2-1ad73064d6ff None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:50.977630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e11a48-7b8e-4420-93c2-1ad73064d6ff None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:50.978172 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e11a48-7b8e-4420-93c2-1ad73064d6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:51.000145 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-79e11a48-7b8e-4420-93c2-1ad73064d6ff None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:16:51.002052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-79e11a48-7b8e-4420-93c2-1ad73064d6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:51.081673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1228/2479] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:16:51.092879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a2248-a56e-4587-91a6-4ea2336a3e5a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:51.096578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-656a2248-a56e-4587-91a6-4ea2336a3e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:51.179194 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a2248-a56e-4587-91a6-4ea2336a3e5a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:51.183273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6c967-4a47-477d-96c5-37aa0acb958a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:51.183900 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6c967-4a47-477d-96c5-37aa0acb958a service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:51.184187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6c967-4a47-477d-96c5-37aa0acb958a service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:51.184491 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6c967-4a47-477d-96c5-37aa0acb958a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:51.195069 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c6c967-4a47-477d-96c5-37aa0acb958a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:51.196683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c6c967-4a47-477d-96c5-37aa0acb958a service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:51.206932 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1252/2480] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:16:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:16:51.664065 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e11a48-7b8e-4420-93c2-1ad73064d6ff None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:51.666251 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-79e11a48-7b8e-4420-93c2-1ad73064d6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:51.702809 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e11a48-7b8e-4420-93c2-1ad73064d6ff None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:51.721415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682e00a-e7b9-4e53-865b-8b3c11e6ca98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:51.721415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682e00a-e7b9-4e53-865b-8b3c11e6ca98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:51.721415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682e00a-e7b9-4e53-865b-8b3c11e6ca98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:51.721415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682e00a-e7b9-4e53-865b-8b3c11e6ca98 admin admin] PATH_INFO: `/v3/projects/b39e710def5f4542b56a21d2b90cdcce` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:51.722291 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682e00a-e7b9-4e53-865b-8b3c11e6ca98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b39e710def5f4542b56a21d2b90cdcce)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:51.725684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682e00a-e7b9-4e53-865b-8b3c11e6ca98 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:51.877032 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2682e00a-e7b9-4e53-865b-8b3c11e6ca98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b39e710def5f4542b56a21d2b90cdcce', 'request_id': 'req-2682e00a-e7b9-4e53-865b-8b3c11e6ca98'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:51.975297 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2682e00a-e7b9-4e53-865b-8b3c11e6ca98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b39e710def5f4542b56a21d2b90cdcce', 'request_id': 'req-2682e00a-e7b9-4e53-865b-8b3c11e6ca98'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:51.983921 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2682e00a-e7b9-4e53-865b-8b3c11e6ca98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b39e710def5f4542b56a21d2b90cdcce', 'request_id': 'req-2682e00a-e7b9-4e53-865b-8b3c11e6ca98'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:51.997206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2682e00a-e7b9-4e53-865b-8b3c11e6ca98 admin admin] The token cache is being invalidate because project b39e710def5f4542b56a21d2b90cdcce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:16:51.997496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2682e00a-e7b9-4e53-865b-8b3c11e6ca98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:51.998588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2682e00a-e7b9-4e53-865b-8b3c11e6ca98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:16:51.999899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1229/2481] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:16:51 2022] DELETE /identity/v3/projects/b39e710def5f4542b56a21d2b90cdcce => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:16:52.005893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab147fd3-eea4-4913-9ff3-b2b0f1719765 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:52.006406 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab147fd3-eea4-4913-9ff3-b2b0f1719765 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:52.013088 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab147fd3-eea4-4913-9ff3-b2b0f1719765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:52.030988 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ab147fd3-eea4-4913-9ff3-b2b0f1719765 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:16:52.032380 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ab147fd3-eea4-4913-9ff3-b2b0f1719765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:52.073131 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:52.116597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1253/2482] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:16:52.125037 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f58b38-d613-4c49-ac3f-dd5dfc7ea987 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:52.125322 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f58b38-d613-4c49-ac3f-dd5dfc7ea987 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:52.125595 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f58b38-d613-4c49-ac3f-dd5dfc7ea987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:52.153797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-71f58b38-d613-4c49-ac3f-dd5dfc7ea987 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:16:52.153797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-71f58b38-d613-4c49-ac3f-dd5dfc7ea987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:52.159485 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1230/2483] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:16:52.166130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab147fd3-eea4-4913-9ff3-b2b0f1719765 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:52.182737 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab147fd3-eea4-4913-9ff3-b2b0f1719765 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:52.189171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a5f8d-1709-450a-9192-856179e75913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:52.190147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a5f8d-1709-450a-9192-856179e75913 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:52.191124 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a5f8d-1709-450a-9192-856179e75913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:52.191579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a5f8d-1709-450a-9192-856179e75913 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:52.192546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5a5f8d-1709-450a-9192-856179e75913 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:16:52.192992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5a5f8d-1709-450a-9192-856179e75913 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:52.194746 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5a5f8d-1709-450a-9192-856179e75913 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:52.210213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1254/2484] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 12:16:52.215174 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f58b38-d613-4c49-ac3f-dd5dfc7ea987 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:52.225438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f58b38-d613-4c49-ac3f-dd5dfc7ea987 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:52.232204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a2b87-e588-4a2a-bfac-e8316772ce19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:52.233044 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a2b87-e588-4a2a-bfac-e8316772ce19 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:52.233431 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a2b87-e588-4a2a-bfac-e8316772ce19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:52.233717 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a2b87-e588-4a2a-bfac-e8316772ce19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:52.234648 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a2b87-e588-4a2a-bfac-e8316772ce19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:52.236660 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a2b87-e588-4a2a-bfac-e8316772ce19 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:52.338722 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1231/2485] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:16:52 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:16:52.339785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a5f8d-1709-450a-9192-856179e75913 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:52.358517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a5f8d-1709-450a-9192-856179e75913 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:52.457389 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebadefe-1b1c-48e1-9bb8-0f12312576a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:52.457389 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebadefe-1b1c-48e1-9bb8-0f12312576a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:52.457389 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebadefe-1b1c-48e1-9bb8-0f12312576a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:52.457389 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebadefe-1b1c-48e1-9bb8-0f12312576a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:52.458124 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebadefe-1b1c-48e1-9bb8-0f12312576a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:52.458124 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebadefe-1b1c-48e1-9bb8-0f12312576a1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:52.743560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1255/2486] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:16:52 2022] POST /identity/v3/users => generated 292 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:16:52.747345 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a2b87-e588-4a2a-bfac-e8316772ce19 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:52.760281 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a2b87-e588-4a2a-bfac-e8316772ce19 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:52.770215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38b18f-e160-4410-b38a-94de4a213426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:52.770215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38b18f-e160-4410-b38a-94de4a213426 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:52.770215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38b18f-e160-4410-b38a-94de4a213426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:52.770215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38b18f-e160-4410-b38a-94de4a213426 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:52.770815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38b18f-e160-4410-b38a-94de4a213426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:52.770815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38b18f-e160-4410-b38a-94de4a213426 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:52.782018 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1232/2487] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:16:52.785975 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebadefe-1b1c-48e1-9bb8-0f12312576a1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:52.796509 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebadefe-1b1c-48e1-9bb8-0f12312576a1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:52.801752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f075ab-9e36-4abc-9730-77ab92fa4d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:52.802410 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f075ab-9e36-4abc-9730-77ab92fa4d8e admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:52.802696 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f075ab-9e36-4abc-9730-77ab92fa4d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:52.803106 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f075ab-9e36-4abc-9730-77ab92fa4d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:52.803762 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f075ab-9e36-4abc-9730-77ab92fa4d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:52.805499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f075ab-9e36-4abc-9730-77ab92fa4d8e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:52.818562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1256/2488] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:16:52.834939 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38b18f-e160-4410-b38a-94de4a213426 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:52.845532 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38b18f-e160-4410-b38a-94de4a213426 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:52.852375 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0666d-a2fe-4a28-aba5-f46c9a6956e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:52.853083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0666d-a2fe-4a28-aba5-f46c9a6956e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:52.853366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0666d-a2fe-4a28-aba5-f46c9a6956e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:52.853667 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0666d-a2fe-4a28-aba5-f46c9a6956e0 admin admin] PATH_INFO: `/v3/projects/3fc25b01187d46b0b6804cd8193593ca/users/6d6f613534434534afd8e5d75e70cf1a/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:52.869414 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0666d-a2fe-4a28-aba5-f46c9a6956e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fc25b01187d46b0b6804cd8193593ca, user_id=6d6f613534434534afd8e5d75e70cf1a, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:52.872253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0666d-a2fe-4a28-aba5-f46c9a6956e0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:52.964985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1233/2489] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:16:52 2022] PUT /identity/v3/projects/3fc25b01187d46b0b6804cd8193593ca/users/6d6f613534434534afd8e5d75e70cf1a/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:16:52.976343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4c922-0d37-4be0-8c6b-94cb14d7ae2d None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:52.976703 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4c922-0d37-4be0-8c6b-94cb14d7ae2d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:52.976983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4c922-0d37-4be0-8c6b-94cb14d7ae2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:53.007361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d6a4c922-0d37-4be0-8c6b-94cb14d7ae2d None None] MFA Rules not processed for user `6d6f613534434534afd8e5d75e70cf1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:16:53.008332 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d6a4c922-0d37-4be0-8c6b-94cb14d7ae2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:53.097485 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1257/2490] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:16:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:16:53.103172 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0666d-a2fe-4a28-aba5-f46c9a6956e0 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:53.128894 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:53.182117 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0666d-a2fe-4a28-aba5-f46c9a6956e0 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:53.191384 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a27e6a-2c9a-4f0c-9a26-ca34a610ccb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:53.193254 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a27e6a-2c9a-4f0c-9a26-ca34a610ccb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:53.193831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a27e6a-2c9a-4f0c-9a26-ca34a610ccb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:53.194369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a27e6a-2c9a-4f0c-9a26-ca34a610ccb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:53.195810 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a27e6a-2c9a-4f0c-9a26-ca34a610ccb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:53.198389 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a27e6a-2c9a-4f0c-9a26-ca34a610ccb3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:53.285521 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1234/2491] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:16:53 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:16:53.290051 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4c922-0d37-4be0-8c6b-94cb14d7ae2d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:53.301298 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4c922-0d37-4be0-8c6b-94cb14d7ae2d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:53.338185 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:53.367775 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89ddbb-c647-4123-91d8-70589d45ea27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:53.368717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89ddbb-c647-4123-91d8-70589d45ea27 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:53.369166 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89ddbb-c647-4123-91d8-70589d45ea27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:53.369486 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89ddbb-c647-4123-91d8-70589d45ea27 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:53.370644 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89ddbb-c647-4123-91d8-70589d45ea27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:53.372646 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89ddbb-c647-4123-91d8-70589d45ea27 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:53.440823 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a27e6a-2c9a-4f0c-9a26-ca34a610ccb3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:53.447884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f3a27e6a-2c9a-4f0c-9a26-ca34a610ccb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:53.487680 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1258/2492] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:16:53 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:16:53.488924 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89ddbb-c647-4123-91d8-70589d45ea27 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:53.500688 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89ddbb-c647-4123-91d8-70589d45ea27 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:53.506940 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce156a-4f43-4c3d-9824-e5bd54472019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:53.507644 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce156a-4f43-4c3d-9824-e5bd54472019 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:53.508015 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce156a-4f43-4c3d-9824-e5bd54472019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:53.508352 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce156a-4f43-4c3d-9824-e5bd54472019 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:53.509067 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce156a-4f43-4c3d-9824-e5bd54472019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:53.510860 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce156a-4f43-4c3d-9824-e5bd54472019 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:53.526577 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1259/2493] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:16:53.527979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce156a-4f43-4c3d-9824-e5bd54472019 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:53.540271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce156a-4f43-4c3d-9824-e5bd54472019 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:53.544832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a27e6a-2c9a-4f0c-9a26-ca34a610ccb3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:53.544990 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b0122-d558-47ec-aafe-6f6ff8478ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:53.545680 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b0122-d558-47ec-aafe-6f6ff8478ccc admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:53.546044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b0122-d558-47ec-aafe-6f6ff8478ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:53.546411 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b0122-d558-47ec-aafe-6f6ff8478ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:53.547122 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b0122-d558-47ec-aafe-6f6ff8478ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:53.548788 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b0122-d558-47ec-aafe-6f6ff8478ccc admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:53.551532 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16820ad5-093f-4aad-a5c7-0e80cd715e2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:53.552150 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16820ad5-093f-4aad-a5c7-0e80cd715e2c service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:53.552439 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16820ad5-093f-4aad-a5c7-0e80cd715e2c service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:53.552715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16820ad5-093f-4aad-a5c7-0e80cd715e2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:53.554583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-16820ad5-093f-4aad-a5c7-0e80cd715e2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:53.572977 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1260/2494] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:16:53.578981 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b0122-d558-47ec-aafe-6f6ff8478ccc admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:53.586096 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b0122-d558-47ec-aafe-6f6ff8478ccc admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:53.591226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6fa68-6a46-45f7-bb7f-340c0f14b91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:53.591896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6fa68-6a46-45f7-bb7f-340c0f14b91d admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:53.592176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6fa68-6a46-45f7-bb7f-340c0f14b91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:53.592451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6fa68-6a46-45f7-bb7f-340c0f14b91d admin admin] PATH_INFO: `/v3/projects/137d61e29d774f7dad974ae2d6917fe8/users/5ad219d54ace4e7caba01c2bf8d13178/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:53.610479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc6fa68-6a46-45f7-bb7f-340c0f14b91d admin admin] RBAC: Authorizing `identity:create_grant(project_id=137d61e29d774f7dad974ae2d6917fe8, user_id=5ad219d54ace4e7caba01c2bf8d13178, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:53.614114 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc6fa68-6a46-45f7-bb7f-340c0f14b91d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:53.667832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16820ad5-093f-4aad-a5c7-0e80cd715e2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:53.669422 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16820ad5-093f-4aad-a5c7-0e80cd715e2c service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:53.689190 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1235/2495] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:16:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:16:53.719651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1261/2496] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:16:53 2022] PUT /identity/v3/projects/137d61e29d774f7dad974ae2d6917fe8/users/5ad219d54ace4e7caba01c2bf8d13178/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:16:53.793569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be7cdc-08e5-43e6-bf04-ac5b19bbe956 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:53.794074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be7cdc-08e5-43e6-bf04-ac5b19bbe956 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:53.794177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be7cdc-08e5-43e6-bf04-ac5b19bbe956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:53.815228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:53.826898 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-90be7cdc-08e5-43e6-bf04-ac5b19bbe956 None None] MFA Rules not processed for user `5ad219d54ace4e7caba01c2bf8d13178`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:16:53.828240 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-90be7cdc-08e5-43e6-bf04-ac5b19bbe956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:53.931180 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1236/2497] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:16:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:16:53.936001 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6fa68-6a46-45f7-bb7f-340c0f14b91d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:54.021780 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6fa68-6a46-45f7-bb7f-340c0f14b91d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:54.028174 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d42f6f-1357-4563-9dd6-525ba00fbbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:54.028897 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d42f6f-1357-4563-9dd6-525ba00fbbd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:54.029662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d42f6f-1357-4563-9dd6-525ba00fbbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:54.030163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d42f6f-1357-4563-9dd6-525ba00fbbd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:54.031248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d42f6f-1357-4563-9dd6-525ba00fbbd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:54.035592 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d42f6f-1357-4563-9dd6-525ba00fbbd6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:54.416698 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1262/2498] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:16:53 2022] POST /identity/v3/projects => generated 385 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:16:54.420851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be7cdc-08e5-43e6-bf04-ac5b19bbe956 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:54.432892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be7cdc-08e5-43e6-bf04-ac5b19bbe956 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:54.497933 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222515c-c5ef-4959-9928-fc80cfaebe80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:54.498643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222515c-c5ef-4959-9928-fc80cfaebe80 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:54.498949 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222515c-c5ef-4959-9928-fc80cfaebe80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:54.499234 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222515c-c5ef-4959-9928-fc80cfaebe80 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:54.499885 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222515c-c5ef-4959-9928-fc80cfaebe80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:54.501814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222515c-c5ef-4959-9928-fc80cfaebe80 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:54.661144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1237/2499] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:16:54 2022] POST /identity/v3/users => generated 299 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:16:54.670006 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d42f6f-1357-4563-9dd6-525ba00fbbd6 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:54.681343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d42f6f-1357-4563-9dd6-525ba00fbbd6 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:54.686090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440333b2-b8ce-4dc2-9d69-986fb62d63da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:54.690909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440333b2-b8ce-4dc2-9d69-986fb62d63da admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:54.690909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440333b2-b8ce-4dc2-9d69-986fb62d63da admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:54.690909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440333b2-b8ce-4dc2-9d69-986fb62d63da admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:54.690909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440333b2-b8ce-4dc2-9d69-986fb62d63da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:54.690909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440333b2-b8ce-4dc2-9d69-986fb62d63da admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:54.701853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1263/2500] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:16:54.708784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222515c-c5ef-4959-9928-fc80cfaebe80 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:54.719820 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222515c-c5ef-4959-9928-fc80cfaebe80 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:54.725253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21b99f-577e-46bb-84e6-7e1d5b73f5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:54.725989 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21b99f-577e-46bb-84e6-7e1d5b73f5f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:54.726257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21b99f-577e-46bb-84e6-7e1d5b73f5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:54.726535 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21b99f-577e-46bb-84e6-7e1d5b73f5f6 admin admin] PATH_INFO: `/v3/projects/49a7a5ca3aee42c4a34030b29d6b564d/users/f8bc08cc2ed84da1985dc9a6bd8d2fea/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:54.750433 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db21b99f-577e-46bb-84e6-7e1d5b73f5f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49a7a5ca3aee42c4a34030b29d6b564d, user_id=f8bc08cc2ed84da1985dc9a6bd8d2fea, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:54.753880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db21b99f-577e-46bb-84e6-7e1d5b73f5f6 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:54.879195 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1238/2501] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:16:54 2022] PUT /identity/v3/projects/49a7a5ca3aee42c4a34030b29d6b564d/users/f8bc08cc2ed84da1985dc9a6bd8d2fea/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:16:54.883529 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440333b2-b8ce-4dc2-9d69-986fb62d63da admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:54.895192 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440333b2-b8ce-4dc2-9d69-986fb62d63da admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:54.950352 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:54.964675 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92936041-c192-4bb7-aeb6-52c1982b3d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:54.965533 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92936041-c192-4bb7-aeb6-52c1982b3d51 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:54.965533 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92936041-c192-4bb7-aeb6-52c1982b3d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:54.966716 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92936041-c192-4bb7-aeb6-52c1982b3d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:54.967370 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92936041-c192-4bb7-aeb6-52c1982b3d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:54.968960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92936041-c192-4bb7-aeb6-52c1982b3d51 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:54.981755 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1264/2502] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:16:54.986756 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21b99f-577e-46bb-84e6-7e1d5b73f5f6 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:54.997994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21b99f-577e-46bb-84e6-7e1d5b73f5f6 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:55.003478 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5814bfa-898b-4337-98fa-1df616ff0201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:55.004144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5814bfa-898b-4337-98fa-1df616ff0201 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:55.004436 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5814bfa-898b-4337-98fa-1df616ff0201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:55.004720 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5814bfa-898b-4337-98fa-1df616ff0201 admin admin] PATH_INFO: `/v3/projects/49a7a5ca3aee42c4a34030b29d6b564d/users/f8bc08cc2ed84da1985dc9a6bd8d2fea/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:55.006691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5814bfa-898b-4337-98fa-1df616ff0201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49a7a5ca3aee42c4a34030b29d6b564d, user_id=f8bc08cc2ed84da1985dc9a6bd8d2fea, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:55.009416 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5814bfa-898b-4337-98fa-1df616ff0201 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:55.022321 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1239/2503] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:16:54 2022] PUT /identity/v3/projects/49a7a5ca3aee42c4a34030b29d6b564d/users/f8bc08cc2ed84da1985dc9a6bd8d2fea/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:16:55.033036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d2c70-bf62-440e-8f48-cad051687107 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:55.033569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d2c70-bf62-440e-8f48-cad051687107 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:55.033569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d2c70-bf62-440e-8f48-cad051687107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:55.063260 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4e6d2c70-bf62-440e-8f48-cad051687107 None None] MFA Rules not processed for user `f8bc08cc2ed84da1985dc9a6bd8d2fea`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:16:55.064561 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4e6d2c70-bf62-440e-8f48-cad051687107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:55.136389 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1265/2504] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:16:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:16:55.148614 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5814bfa-898b-4337-98fa-1df616ff0201 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:55.150499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f5814bfa-898b-4337-98fa-1df616ff0201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:16:55.211782 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5814bfa-898b-4337-98fa-1df616ff0201 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:55.215437 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93735141-c89f-46e7-b0ef-9c45930d82c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:55.216070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93735141-c89f-46e7-b0ef-9c45930d82c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:55.216331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93735141-c89f-46e7-b0ef-9c45930d82c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:55.216608 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93735141-c89f-46e7-b0ef-9c45930d82c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:55.221775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:16:55.285574 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93735141-c89f-46e7-b0ef-9c45930d82c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:55.287042 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93735141-c89f-46e7-b0ef-9c45930d82c2 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:55.297247 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1240/2505] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:16:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:16:58.440102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d2c70-bf62-440e-8f48-cad051687107 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:16:58.450729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d2c70-bf62-440e-8f48-cad051687107 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:16:58.454323 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417dacc0-918d-46b1-90fb-192e93563b3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:16:58.454918 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dacc0-918d-46b1-90fb-192e93563b3f service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:16:58.455198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dacc0-918d-46b1-90fb-192e93563b3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:16:58.455587 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dacc0-918d-46b1-90fb-192e93563b3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:16:58.513747 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417dacc0-918d-46b1-90fb-192e93563b3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:16:58.515211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417dacc0-918d-46b1-90fb-192e93563b3f service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:16:58.524901 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1266/2506] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:16:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:17:01.025515 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93735141-c89f-46e7-b0ef-9c45930d82c2 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:01.027311 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-93735141-c89f-46e7-b0ef-9c45930d82c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:01.039633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93735141-c89f-46e7-b0ef-9c45930d82c2 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:01.043788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680ac6d-c1d3-4670-bc2a-948d9b431daf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:01.044412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680ac6d-c1d3-4670-bc2a-948d9b431daf service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:01.044697 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680ac6d-c1d3-4670-bc2a-948d9b431daf service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:01.044979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680ac6d-c1d3-4670-bc2a-948d9b431daf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:01.055762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680ac6d-c1d3-4670-bc2a-948d9b431daf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:01.057282 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680ac6d-c1d3-4670-bc2a-948d9b431daf service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:01.069314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1241/2507] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:17:01 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:17:04.608962 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417dacc0-918d-46b1-90fb-192e93563b3f service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:04.610486 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-417dacc0-918d-46b1-90fb-192e93563b3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:04.687815 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417dacc0-918d-46b1-90fb-192e93563b3f service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:04.694044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034579f-7366-4f0b-a4c7-968027393fb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:04.694602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034579f-7366-4f0b-a4c7-968027393fb6 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:04.694889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034579f-7366-4f0b-a4c7-968027393fb6 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:04.695172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034579f-7366-4f0b-a4c7-968027393fb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:04.697001 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0034579f-7366-4f0b-a4c7-968027393fb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:04.706418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:04.710982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034579f-7366-4f0b-a4c7-968027393fb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:04.712687 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034579f-7366-4f0b-a4c7-968027393fb6 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:04.724026 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1267/2508] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:17:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:17:05.399388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680ac6d-c1d3-4670-bc2a-948d9b431daf service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:05.401093 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8680ac6d-c1d3-4670-bc2a-948d9b431daf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:05.476349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680ac6d-c1d3-4670-bc2a-948d9b431daf service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:05.480295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad430262-449e-4595-bac3-25e3613c7874 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:05.480972 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad430262-449e-4595-bac3-25e3613c7874 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:05.481323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad430262-449e-4595-bac3-25e3613c7874 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:05.482370 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad430262-449e-4595-bac3-25e3613c7874 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:05.484237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ad430262-449e-4595-bac3-25e3613c7874 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:05.494558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad430262-449e-4595-bac3-25e3613c7874 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:05.496049 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad430262-449e-4595-bac3-25e3613c7874 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:05.504951 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1242/2509] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:17:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:17:06.987618 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034579f-7366-4f0b-a4c7-968027393fb6 service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:07.064580 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034579f-7366-4f0b-a4c7-968027393fb6 service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:07.069903 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985cf66-4bb5-40dd-9c5a-94a7c51acced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:07.070397 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985cf66-4bb5-40dd-9c5a-94a7c51acced admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:07.070708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985cf66-4bb5-40dd-9c5a-94a7c51acced admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:07.070990 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985cf66-4bb5-40dd-9c5a-94a7c51acced admin admin] PATH_INFO: `/v3/users/6d6f613534434534afd8e5d75e70cf1a` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:07.072192 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5985cf66-4bb5-40dd-9c5a-94a7c51acced admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d6f613534434534afd8e5d75e70cf1a)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:07.074290 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5985cf66-4bb5-40dd-9c5a-94a7c51acced admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:07.286966 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad7bf9-fedd-4d76-b1e1-417e60ecf623 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:07.287461 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad7bf9-fedd-4d76-b1e1-417e60ecf623 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:07.287461 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad7bf9-fedd-4d76-b1e1-417e60ecf623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:07.313136 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e8ad7bf9-fedd-4d76-b1e1-417e60ecf623 None None] MFA Rules not processed for user `d76a164ed7e14d9eb2db016b68a84d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:07.314597 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e8ad7bf9-fedd-4d76-b1e1-417e60ecf623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:07.333651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5985cf66-4bb5-40dd-9c5a-94a7c51acced admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d6f613534434534afd8e5d75e70cf1a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:07.348354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5985cf66-4bb5-40dd-9c5a-94a7c51acced admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d6f613534434534afd8e5d75e70cf1a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:07.356510 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5985cf66-4bb5-40dd-9c5a-94a7c51acced admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d6f613534434534afd8e5d75e70cf1a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:07.384168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1243/2510] 10.222.0.62 () {58 vars in 967 bytes} [Wed Sep 14 12:17:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:17:07.398218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad7bf9-fedd-4d76-b1e1-417e60ecf623 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:07.399987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e8ad7bf9-fedd-4d76-b1e1-417e60ecf623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:07.443734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5985cf66-4bb5-40dd-9c5a-94a7c51acced admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d6f613534434534afd8e5d75e70cf1a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:07.444686 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5985cf66-4bb5-40dd-9c5a-94a7c51acced admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d6f613534434534afd8e5d75e70cf1a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:07.452977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:07.455780 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1268/2511] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:17:06 2022] DELETE /identity/v3/users/6d6f613534434534afd8e5d75e70cf1a => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:07.461915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f5cd0-610f-4b53-999d-c8eb79c17776 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:07.462430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f5cd0-610f-4b53-999d-c8eb79c17776 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:07.463285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f5cd0-610f-4b53-999d-c8eb79c17776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:07.483256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad7bf9-fedd-4d76-b1e1-417e60ecf623 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:07.490239 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1f5f5cd0-610f-4b53-999d-c8eb79c17776 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:07.491738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1f5f5cd0-610f-4b53-999d-c8eb79c17776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:07.554923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082d50f-c07e-4d9e-b7ec-3b089d316661 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1047c7d18c1466d989cd895117cd807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:07.555549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082d50f-c07e-4d9e-b7ec-3b089d316661 service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:07.555895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082d50f-c07e-4d9e-b7ec-3b089d316661 service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:07.556117 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082d50f-c07e-4d9e-b7ec-3b089d316661 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:07.557976 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1082d50f-c07e-4d9e-b7ec-3b089d316661 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:07.561075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1269/2512] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:07.574964 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f5cd0-610f-4b53-999d-c8eb79c17776 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:07.576386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1f5f5cd0-610f-4b53-999d-c8eb79c17776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:07.594485 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:07.627900 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1082d50f-c07e-4d9e-b7ec-3b089d316661 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:07.630331 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1082d50f-c07e-4d9e-b7ec-3b089d316661 service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:07.642415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1244/2513] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 14 12:17:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 14 12:17:07.647784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f5cd0-610f-4b53-999d-c8eb79c17776 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:07.652520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1dce84-b441-4d59-886b-8ee12ec2695b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:07.653370 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dce84-b441-4d59-886b-8ee12ec2695b service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:07.653671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dce84-b441-4d59-886b-8ee12ec2695b service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:07.654046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dce84-b441-4d59-886b-8ee12ec2695b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:07.664908 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1dce84-b441-4d59-886b-8ee12ec2695b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:07.667663 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1dce84-b441-4d59-886b-8ee12ec2695b service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:07.682296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1270/2514] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:17:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:17:08.136530 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082d50f-c07e-4d9e-b7ec-3b089d316661 service swift] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:08.138462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1082d50f-c07e-4d9e-b7ec-3b089d316661 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:08.160715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082d50f-c07e-4d9e-b7ec-3b089d316661 service swift] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:08.165678 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651d418-fba9-43c1-a7d3-74f37f7ab8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:08.166362 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651d418-fba9-43c1-a7d3-74f37f7ab8e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:08.166648 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651d418-fba9-43c1-a7d3-74f37f7ab8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:08.166928 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651d418-fba9-43c1-a7d3-74f37f7ab8e8 admin admin] PATH_INFO: `/v3/projects/3fc25b01187d46b0b6804cd8193593ca` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:08.168059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651d418-fba9-43c1-a7d3-74f37f7ab8e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fc25b01187d46b0b6804cd8193593ca)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:08.170091 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651d418-fba9-43c1-a7d3-74f37f7ab8e8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:08.333789 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d651d418-fba9-43c1-a7d3-74f37f7ab8e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fc25b01187d46b0b6804cd8193593ca', 'request_id': 'req-d651d418-fba9-43c1-a7d3-74f37f7ab8e8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:08.442444 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d651d418-fba9-43c1-a7d3-74f37f7ab8e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fc25b01187d46b0b6804cd8193593ca', 'request_id': 'req-d651d418-fba9-43c1-a7d3-74f37f7ab8e8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:08.450366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d651d418-fba9-43c1-a7d3-74f37f7ab8e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fc25b01187d46b0b6804cd8193593ca', 'request_id': 'req-d651d418-fba9-43c1-a7d3-74f37f7ab8e8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:08.463927 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d651d418-fba9-43c1-a7d3-74f37f7ab8e8 admin admin] The token cache is being invalidate because project 3fc25b01187d46b0b6804cd8193593ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:17:08.464378 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d651d418-fba9-43c1-a7d3-74f37f7ab8e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:08.465104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d651d418-fba9-43c1-a7d3-74f37f7ab8e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:08.467780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1245/2515] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:17:08 2022] DELETE /identity/v3/projects/3fc25b01187d46b0b6804cd8193593ca => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:08.472285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1dce84-b441-4d59-886b-8ee12ec2695b service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:08.483209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1dce84-b441-4d59-886b-8ee12ec2695b service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:08.560401 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928895fb-5069-4fba-a895-0a0ead500d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:08.561068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928895fb-5069-4fba-a895-0a0ead500d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:08.561355 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928895fb-5069-4fba-a895-0a0ead500d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:08.561639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928895fb-5069-4fba-a895-0a0ead500d1c admin admin] PATH_INFO: `/v3/users/5ad219d54ace4e7caba01c2bf8d13178` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:08.562810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928895fb-5069-4fba-a895-0a0ead500d1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ad219d54ace4e7caba01c2bf8d13178)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:08.564735 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928895fb-5069-4fba-a895-0a0ead500d1c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:08.755425 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-928895fb-5069-4fba-a895-0a0ead500d1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ad219d54ace4e7caba01c2bf8d13178'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:08.768618 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-928895fb-5069-4fba-a895-0a0ead500d1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ad219d54ace4e7caba01c2bf8d13178'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:08.776519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-928895fb-5069-4fba-a895-0a0ead500d1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ad219d54ace4e7caba01c2bf8d13178'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:08.831898 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-928895fb-5069-4fba-a895-0a0ead500d1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ad219d54ace4e7caba01c2bf8d13178'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:08.832560 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-928895fb-5069-4fba-a895-0a0ead500d1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ad219d54ace4e7caba01c2bf8d13178'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:08.842107 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1271/2516] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:17:08 2022] DELETE /identity/v3/users/5ad219d54ace4e7caba01c2bf8d13178 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:09.239794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651d418-fba9-43c1-a7d3-74f37f7ab8e8 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:09.241293 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d651d418-fba9-43c1-a7d3-74f37f7ab8e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:09.255220 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651d418-fba9-43c1-a7d3-74f37f7ab8e8 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:09.259167 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:09.317687 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f13e53-89c8-4ee3-8a28-aca27fbda147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:09.318583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f13e53-89c8-4ee3-8a28-aca27fbda147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:09.318716 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f13e53-89c8-4ee3-8a28-aca27fbda147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:09.318896 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f13e53-89c8-4ee3-8a28-aca27fbda147 admin admin] PATH_INFO: `/v3/projects/137d61e29d774f7dad974ae2d6917fe8` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:09.320108 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f13e53-89c8-4ee3-8a28-aca27fbda147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=137d61e29d774f7dad974ae2d6917fe8)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:09.322128 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f13e53-89c8-4ee3-8a28-aca27fbda147 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:09.438801 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-76f13e53-89c8-4ee3-8a28-aca27fbda147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '137d61e29d774f7dad974ae2d6917fe8', 'request_id': 'req-76f13e53-89c8-4ee3-8a28-aca27fbda147'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:09.505280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-76f13e53-89c8-4ee3-8a28-aca27fbda147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '137d61e29d774f7dad974ae2d6917fe8', 'request_id': 'req-76f13e53-89c8-4ee3-8a28-aca27fbda147'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:09.511088 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-76f13e53-89c8-4ee3-8a28-aca27fbda147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '137d61e29d774f7dad974ae2d6917fe8', 'request_id': 'req-76f13e53-89c8-4ee3-8a28-aca27fbda147'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:09.522767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-76f13e53-89c8-4ee3-8a28-aca27fbda147 admin admin] The token cache is being invalidate because project 137d61e29d774f7dad974ae2d6917fe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:17:09.523332 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-76f13e53-89c8-4ee3-8a28-aca27fbda147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:09.523970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-76f13e53-89c8-4ee3-8a28-aca27fbda147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:09.525523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1246/2517] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:17:09 2022] DELETE /identity/v3/projects/137d61e29d774f7dad974ae2d6917fe8 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:09.529566 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928895fb-5069-4fba-a895-0a0ead500d1c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:09.531774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-928895fb-5069-4fba-a895-0a0ead500d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:09.544187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928895fb-5069-4fba-a895-0a0ead500d1c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:09.577052 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:09.601680 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa88346-b8af-463f-bfe2-bf58d4d872e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:09.602335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa88346-b8af-463f-bfe2-bf58d4d872e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:09.602621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa88346-b8af-463f-bfe2-bf58d4d872e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:09.602905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa88346-b8af-463f-bfe2-bf58d4d872e5 admin admin] PATH_INFO: `/v3/users/f8bc08cc2ed84da1985dc9a6bd8d2fea` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:09.604019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa88346-b8af-463f-bfe2-bf58d4d872e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8bc08cc2ed84da1985dc9a6bd8d2fea)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:09.605955 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa88346-b8af-463f-bfe2-bf58d4d872e5 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:09.840148 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1aa88346-b8af-463f-bfe2-bf58d4d872e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8bc08cc2ed84da1985dc9a6bd8d2fea'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:09.857163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1aa88346-b8af-463f-bfe2-bf58d4d872e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8bc08cc2ed84da1985dc9a6bd8d2fea'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:09.866437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1aa88346-b8af-463f-bfe2-bf58d4d872e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8bc08cc2ed84da1985dc9a6bd8d2fea'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:09.951809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1aa88346-b8af-463f-bfe2-bf58d4d872e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8bc08cc2ed84da1985dc9a6bd8d2fea'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:09.952534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1aa88346-b8af-463f-bfe2-bf58d4d872e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8bc08cc2ed84da1985dc9a6bd8d2fea'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:09.959942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1272/2518] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:17:09 2022] DELETE /identity/v3/users/f8bc08cc2ed84da1985dc9a6bd8d2fea => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:10.393035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f13e53-89c8-4ee3-8a28-aca27fbda147 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:10.395318 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-76f13e53-89c8-4ee3-8a28-aca27fbda147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:10.409147 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f13e53-89c8-4ee3-8a28-aca27fbda147 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:10.467513 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1c15d-c456-4b9d-a516-de792e4581d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:10.468159 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1c15d-c456-4b9d-a516-de792e4581d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:10.468478 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1c15d-c456-4b9d-a516-de792e4581d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:10.468685 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1c15d-c456-4b9d-a516-de792e4581d4 admin admin] PATH_INFO: `/v3/projects/49a7a5ca3aee42c4a34030b29d6b564d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:10.469800 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1c15d-c456-4b9d-a516-de792e4581d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49a7a5ca3aee42c4a34030b29d6b564d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:10.471739 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1c15d-c456-4b9d-a516-de792e4581d4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:10.586575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:10.593518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a3c1c15d-c456-4b9d-a516-de792e4581d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49a7a5ca3aee42c4a34030b29d6b564d', 'request_id': 'req-a3c1c15d-c456-4b9d-a516-de792e4581d4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:10.694406 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a3c1c15d-c456-4b9d-a516-de792e4581d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49a7a5ca3aee42c4a34030b29d6b564d', 'request_id': 'req-a3c1c15d-c456-4b9d-a516-de792e4581d4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:10.702760 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a3c1c15d-c456-4b9d-a516-de792e4581d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49a7a5ca3aee42c4a34030b29d6b564d', 'request_id': 'req-a3c1c15d-c456-4b9d-a516-de792e4581d4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:10.723188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a3c1c15d-c456-4b9d-a516-de792e4581d4 admin admin] The token cache is being invalidate because project 49a7a5ca3aee42c4a34030b29d6b564d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:17:10.723188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a3c1c15d-c456-4b9d-a516-de792e4581d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:10.723188 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a3c1c15d-c456-4b9d-a516-de792e4581d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:10.725504 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1247/2519] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:17:10 2022] DELETE /identity/v3/projects/49a7a5ca3aee42c4a34030b29d6b564d => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:10.730001 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53d80e-a3c9-4b47-a318-3a9b534ca279 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:10.730479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53d80e-a3c9-4b47-a318-3a9b534ca279 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:10.730617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53d80e-a3c9-4b47-a318-3a9b534ca279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:10.751790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ad53d80e-a3c9-4b47-a318-3a9b534ca279 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:10.753119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ad53d80e-a3c9-4b47-a318-3a9b534ca279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:10.826811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1273/2520] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:10.836043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fb550-2246-4a05-a2f9-ff245d12c473 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:10.836487 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fb550-2246-4a05-a2f9-ff245d12c473 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:10.837414 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fb550-2246-4a05-a2f9-ff245d12c473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:10.859899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9a0fb550-2246-4a05-a2f9-ff245d12c473 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:10.861272 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9a0fb550-2246-4a05-a2f9-ff245d12c473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:10.870314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1248/2521] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:10.874616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53d80e-a3c9-4b47-a318-3a9b534ca279 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:10.884364 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53d80e-a3c9-4b47-a318-3a9b534ca279 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:10.888971 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a51458-6987-4199-8881-ff4d92dc7d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:10.889591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a51458-6987-4199-8881-ff4d92dc7d3a admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:10.889910 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a51458-6987-4199-8881-ff4d92dc7d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:10.890198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a51458-6987-4199-8881-ff4d92dc7d3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:10.890846 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a51458-6987-4199-8881-ff4d92dc7d3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:17:10.891156 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a51458-6987-4199-8881-ff4d92dc7d3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:10.892791 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a51458-6987-4199-8881-ff4d92dc7d3a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:10.904907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1274/2522] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:17:10.909373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fb550-2246-4a05-a2f9-ff245d12c473 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:10.919374 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fb550-2246-4a05-a2f9-ff245d12c473 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:10.924082 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0ed17-e85a-4364-90b1-f5d6748e6dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:10.924707 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0ed17-e85a-4364-90b1-f5d6748e6dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:10.924995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0ed17-e85a-4364-90b1-f5d6748e6dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:10.925274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0ed17-e85a-4364-90b1-f5d6748e6dd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:10.927141 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0ed17-e85a-4364-90b1-f5d6748e6dd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:10.928673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0ed17-e85a-4364-90b1-f5d6748e6dd3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:11.022126 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1249/2523] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:17:10 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:17:11.027057 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a51458-6987-4199-8881-ff4d92dc7d3a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:11.041083 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a51458-6987-4199-8881-ff4d92dc7d3a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:11.081346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:11.098278 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802a240-11cd-42f9-becb-3d1372c2882e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:11.098940 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802a240-11cd-42f9-becb-3d1372c2882e admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:11.099220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802a240-11cd-42f9-becb-3d1372c2882e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:11.099587 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802a240-11cd-42f9-becb-3d1372c2882e admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:11.100256 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802a240-11cd-42f9-becb-3d1372c2882e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:11.102202 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802a240-11cd-42f9-becb-3d1372c2882e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:11.214936 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1275/2524] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:17:11 2022] POST /identity/v3/users => generated 288 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:17:11.219106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0ed17-e85a-4364-90b1-f5d6748e6dd3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:11.230187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0ed17-e85a-4364-90b1-f5d6748e6dd3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:11.234997 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7ffbd-2460-47b3-8d7e-c4760d4cf2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:11.235605 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7ffbd-2460-47b3-8d7e-c4760d4cf2f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:11.235894 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7ffbd-2460-47b3-8d7e-c4760d4cf2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:11.236191 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7ffbd-2460-47b3-8d7e-c4760d4cf2f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:11.236843 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7ffbd-2460-47b3-8d7e-c4760d4cf2f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:11.238499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7ffbd-2460-47b3-8d7e-c4760d4cf2f5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:11.248721 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1250/2525] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:17:11.252658 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802a240-11cd-42f9-becb-3d1372c2882e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:11.262554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802a240-11cd-42f9-becb-3d1372c2882e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:11.267456 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d520b-01bb-4865-8f39-2845bdb29eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:11.268055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d520b-01bb-4865-8f39-2845bdb29eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:11.268345 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d520b-01bb-4865-8f39-2845bdb29eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:11.268623 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d520b-01bb-4865-8f39-2845bdb29eb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:11.269266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d520b-01bb-4865-8f39-2845bdb29eb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:11.271415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d520b-01bb-4865-8f39-2845bdb29eb3 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:11.283806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1276/2526] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:17:11.288525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7ffbd-2460-47b3-8d7e-c4760d4cf2f5 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:11.298403 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7ffbd-2460-47b3-8d7e-c4760d4cf2f5 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:11.303173 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cdbaf-859b-4117-ae85-cb5372859daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:11.303814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cdbaf-859b-4117-ae85-cb5372859daf admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:11.304092 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cdbaf-859b-4117-ae85-cb5372859daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:11.304368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cdbaf-859b-4117-ae85-cb5372859daf admin admin] PATH_INFO: `/v3/projects/61ea049a349f4941becb47bd998a7576/users/b347b58f90b041d5b4e22b1e9684ff61/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:11.319252 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977cdbaf-859b-4117-ae85-cb5372859daf admin admin] RBAC: Authorizing `identity:create_grant(project_id=61ea049a349f4941becb47bd998a7576, user_id=b347b58f90b041d5b4e22b1e9684ff61, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:11.321947 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977cdbaf-859b-4117-ae85-cb5372859daf admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:11.385939 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1251/2527] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:17:11 2022] PUT /identity/v3/projects/61ea049a349f4941becb47bd998a7576/users/b347b58f90b041d5b4e22b1e9684ff61/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:11.396889 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6795d1-a112-4057-8526-528c63fa5232 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:11.398375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6795d1-a112-4057-8526-528c63fa5232 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:11.398375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6795d1-a112-4057-8526-528c63fa5232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:11.427467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3a6795d1-a112-4057-8526-528c63fa5232 None None] MFA Rules not processed for user `b347b58f90b041d5b4e22b1e9684ff61`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:11.428693 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3a6795d1-a112-4057-8526-528c63fa5232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:11.612313 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1277/2528] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:11.623033 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cdbaf-859b-4117-ae85-cb5372859daf admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:11.625040 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-977cdbaf-859b-4117-ae85-cb5372859daf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:11.682147 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cdbaf-859b-4117-ae85-cb5372859daf admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:11.685986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab83d16-2483-442d-a29c-ea859ac9c285 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:11.686603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab83d16-2483-442d-a29c-ea859ac9c285 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:11.686888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab83d16-2483-442d-a29c-ea859ac9c285 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:11.687171 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab83d16-2483-442d-a29c-ea859ac9c285 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:11.698037 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab83d16-2483-442d-a29c-ea859ac9c285 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:11.699480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab83d16-2483-442d-a29c-ea859ac9c285 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:11.709025 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1252/2529] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:17:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:17:21.609280 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=47,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:21.704287 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=63,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:24.063877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6795d1-a112-4057-8526-528c63fa5232 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:24.125988 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6795d1-a112-4057-8526-528c63fa5232 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:24.130571 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61ea5c-aa18-4aaa-93a3-8d74eb15cb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:24.131203 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61ea5c-aa18-4aaa-93a3-8d74eb15cb2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:24.131534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61ea5c-aa18-4aaa-93a3-8d74eb15cb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:24.132106 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61ea5c-aa18-4aaa-93a3-8d74eb15cb2f admin admin] PATH_INFO: `/v3/users/b347b58f90b041d5b4e22b1e9684ff61` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:24.133460 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61ea5c-aa18-4aaa-93a3-8d74eb15cb2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b347b58f90b041d5b4e22b1e9684ff61)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:24.135422 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61ea5c-aa18-4aaa-93a3-8d74eb15cb2f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:24.307782 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ac61ea5c-aa18-4aaa-93a3-8d74eb15cb2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b347b58f90b041d5b4e22b1e9684ff61'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:24.322451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ac61ea5c-aa18-4aaa-93a3-8d74eb15cb2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b347b58f90b041d5b4e22b1e9684ff61'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:24.329656 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ac61ea5c-aa18-4aaa-93a3-8d74eb15cb2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b347b58f90b041d5b4e22b1e9684ff61'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:24.391583 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ac61ea5c-aa18-4aaa-93a3-8d74eb15cb2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b347b58f90b041d5b4e22b1e9684ff61'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:24.392273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ac61ea5c-aa18-4aaa-93a3-8d74eb15cb2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b347b58f90b041d5b4e22b1e9684ff61'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:24.399264 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1278/2530] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:17:24 2022] DELETE /identity/v3/users/b347b58f90b041d5b4e22b1e9684ff61 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:24.403807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889511a-5734-4a24-a5c0-35bebc4b553b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:24.404304 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889511a-5734-4a24-a5c0-35bebc4b553b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:24.404423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889511a-5734-4a24-a5c0-35bebc4b553b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:24.424173 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-0889511a-5734-4a24-a5c0-35bebc4b553b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:24.430927 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0889511a-5734-4a24-a5c0-35bebc4b553b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:24.492923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1253/2531] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:24.503095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61ea5c-aa18-4aaa-93a3-8d74eb15cb2f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:24.504511 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ac61ea5c-aa18-4aaa-93a3-8d74eb15cb2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:24.561843 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61ea5c-aa18-4aaa-93a3-8d74eb15cb2f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:24.565409 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103fb0fc-7010-4d88-adf2-88e4c8f75b53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:24.566043 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103fb0fc-7010-4d88-adf2-88e4c8f75b53 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:24.566330 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103fb0fc-7010-4d88-adf2-88e4c8f75b53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:24.566743 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103fb0fc-7010-4d88-adf2-88e4c8f75b53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:24.576631 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103fb0fc-7010-4d88-adf2-88e4c8f75b53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:24.578085 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103fb0fc-7010-4d88-adf2-88e4c8f75b53 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:24.587416 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1279/2532] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:17:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:17:24.998880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889511a-5734-4a24-a5c0-35bebc4b553b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:25.000416 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0889511a-5734-4a24-a5c0-35bebc4b553b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:25.012777 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889511a-5734-4a24-a5c0-35bebc4b553b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:25.017689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbec9c-b7d8-40b8-98f1-71a53b461547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:25.018388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbec9c-b7d8-40b8-98f1-71a53b461547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:25.018681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbec9c-b7d8-40b8-98f1-71a53b461547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:25.018964 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbec9c-b7d8-40b8-98f1-71a53b461547 admin admin] PATH_INFO: `/v3/projects/61ea049a349f4941becb47bd998a7576` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:25.020102 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbec9c-b7d8-40b8-98f1-71a53b461547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61ea049a349f4941becb47bd998a7576)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:25.022283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbec9c-b7d8-40b8-98f1-71a53b461547 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:25.148400 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d4dbec9c-b7d8-40b8-98f1-71a53b461547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61ea049a349f4941becb47bd998a7576', 'request_id': 'req-d4dbec9c-b7d8-40b8-98f1-71a53b461547'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:25.214590 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d4dbec9c-b7d8-40b8-98f1-71a53b461547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61ea049a349f4941becb47bd998a7576', 'request_id': 'req-d4dbec9c-b7d8-40b8-98f1-71a53b461547'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:25.220467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d4dbec9c-b7d8-40b8-98f1-71a53b461547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61ea049a349f4941becb47bd998a7576', 'request_id': 'req-d4dbec9c-b7d8-40b8-98f1-71a53b461547'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:25.232127 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d4dbec9c-b7d8-40b8-98f1-71a53b461547 admin admin] The token cache is being invalidate because project 61ea049a349f4941becb47bd998a7576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:17:25.232433 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d4dbec9c-b7d8-40b8-98f1-71a53b461547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:25.233057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d4dbec9c-b7d8-40b8-98f1-71a53b461547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:25.234565 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1254/2533] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:17:24 2022] DELETE /identity/v3/projects/61ea049a349f4941becb47bd998a7576 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:30.538031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103fb0fc-7010-4d88-adf2-88e4c8f75b53 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:30.539466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-103fb0fc-7010-4d88-adf2-88e4c8f75b53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:30.570133 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:30.650091 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103fb0fc-7010-4d88-adf2-88e4c8f75b53 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:30.731541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38fdbad-f9f4-4193-9c88-27b9f2ca7e5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:30.732181 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38fdbad-f9f4-4193-9c88-27b9f2ca7e5d service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:30.732532 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38fdbad-f9f4-4193-9c88-27b9f2ca7e5d service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:30.732920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38fdbad-f9f4-4193-9c88-27b9f2ca7e5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:30.735022 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e38fdbad-f9f4-4193-9c88-27b9f2ca7e5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:30.750720 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38fdbad-f9f4-4193-9c88-27b9f2ca7e5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:30.752350 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38fdbad-f9f4-4193-9c88-27b9f2ca7e5d service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:30.764999 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1280/2534] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:17:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:17:35.229059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:39.226454 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe45229-1239-4a45-81ed-0e529078ce30 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:39.227072 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe45229-1239-4a45-81ed-0e529078ce30 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:39.227072 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe45229-1239-4a45-81ed-0e529078ce30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:39.246042 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-dbe45229-1239-4a45-81ed-0e529078ce30 None None] MFA Rules not processed for user `31f3271adc36406fb303fb169020e740`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:39.247442 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-dbe45229-1239-4a45-81ed-0e529078ce30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:39.304669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1255/2535] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:40.826044 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=52 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:41.068292 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38fdbad-f9f4-4193-9c88-27b9f2ca7e5d service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:41.069702 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e38fdbad-f9f4-4193-9c88-27b9f2ca7e5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:41.125514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38fdbad-f9f4-4193-9c88-27b9f2ca7e5d service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:41.129116 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604b026-0a8c-4935-808b-ce6ab3e6ee55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:41.129719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604b026-0a8c-4935-808b-ce6ab3e6ee55 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:41.130046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604b026-0a8c-4935-808b-ce6ab3e6ee55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:41.130323 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604b026-0a8c-4935-808b-ce6ab3e6ee55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:41.132037 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1604b026-0a8c-4935-808b-ce6ab3e6ee55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:41.196412 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604b026-0a8c-4935-808b-ce6ab3e6ee55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:41.200381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604b026-0a8c-4935-808b-ce6ab3e6ee55 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:41.210683 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1281/2536] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:17:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:17:44.945809 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe45229-1239-4a45-81ed-0e529078ce30 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:44.947413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-dbe45229-1239-4a45-81ed-0e529078ce30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:44.962418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe45229-1239-4a45-81ed-0e529078ce30 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:44.967215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c24246-33ca-413d-a026-01947e7777b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:44.967852 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c24246-33ca-413d-a026-01947e7777b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:44.968139 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c24246-33ca-413d-a026-01947e7777b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:44.968424 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c24246-33ca-413d-a026-01947e7777b9 admin admin] PATH_INFO: `/v3/users/2ba1860ce5504c27816a504776463d9c` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:44.969549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c24246-33ca-413d-a026-01947e7777b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ba1860ce5504c27816a504776463d9c)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:44.971505 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c24246-33ca-413d-a026-01947e7777b9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:45.136785 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c2c24246-33ca-413d-a026-01947e7777b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ba1860ce5504c27816a504776463d9c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:45.149322 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c2c24246-33ca-413d-a026-01947e7777b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ba1860ce5504c27816a504776463d9c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:45.156230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c2c24246-33ca-413d-a026-01947e7777b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ba1860ce5504c27816a504776463d9c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:45.253714 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c2c24246-33ca-413d-a026-01947e7777b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ba1860ce5504c27816a504776463d9c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:45.254361 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c2c24246-33ca-413d-a026-01947e7777b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ba1860ce5504c27816a504776463d9c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:45.260795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1256/2537] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:17:44 2022] DELETE /identity/v3/users/2ba1860ce5504c27816a504776463d9c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:45.518069 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604b026-0a8c-4935-808b-ce6ab3e6ee55 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:45.546991 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604b026-0a8c-4935-808b-ce6ab3e6ee55 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:45.586537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e825b9-2b5d-4c8a-871d-188290c29651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:45.594816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e825b9-2b5d-4c8a-871d-188290c29651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:45.594816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e825b9-2b5d-4c8a-871d-188290c29651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:45.594816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e825b9-2b5d-4c8a-871d-188290c29651 admin admin] PATH_INFO: `/v3/projects/e35c2ab9a9644cadadd4da9a544b944b` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:45.594816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e825b9-2b5d-4c8a-871d-188290c29651 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e35c2ab9a9644cadadd4da9a544b944b)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:45.594816 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e825b9-2b5d-4c8a-871d-188290c29651 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:45.698165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a6e825b9-2b5d-4c8a-871d-188290c29651 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e35c2ab9a9644cadadd4da9a544b944b', 'request_id': 'req-a6e825b9-2b5d-4c8a-871d-188290c29651'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:45.701370 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:45.785331 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a6e825b9-2b5d-4c8a-871d-188290c29651 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e35c2ab9a9644cadadd4da9a544b944b', 'request_id': 'req-a6e825b9-2b5d-4c8a-871d-188290c29651'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:45.792170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a6e825b9-2b5d-4c8a-871d-188290c29651 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e35c2ab9a9644cadadd4da9a544b944b', 'request_id': 'req-a6e825b9-2b5d-4c8a-871d-188290c29651'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:45.803040 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a6e825b9-2b5d-4c8a-871d-188290c29651 admin admin] The token cache is being invalidate because project e35c2ab9a9644cadadd4da9a544b944b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:17:45.803327 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a6e825b9-2b5d-4c8a-871d-188290c29651 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:45.803956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a6e825b9-2b5d-4c8a-871d-188290c29651 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:45.805442 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1282/2538] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:17:45 2022] DELETE /identity/v3/projects/e35c2ab9a9644cadadd4da9a544b944b => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:45.810583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247092d-3e42-4839-a732-b7fffa3504ea None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:45.810877 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247092d-3e42-4839-a732-b7fffa3504ea None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:45.811175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247092d-3e42-4839-a732-b7fffa3504ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:45.832683 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4247092d-3e42-4839-a732-b7fffa3504ea None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:45.833982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4247092d-3e42-4839-a732-b7fffa3504ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:45.894452 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1257/2539] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:45.903091 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63856b13-4211-44a1-9947-8e96d125f318 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:45.903409 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63856b13-4211-44a1-9947-8e96d125f318 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:45.903690 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63856b13-4211-44a1-9947-8e96d125f318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:45.923070 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-63856b13-4211-44a1-9947-8e96d125f318 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:45.924282 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-63856b13-4211-44a1-9947-8e96d125f318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:45.932176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1283/2540] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:45.936494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247092d-3e42-4839-a732-b7fffa3504ea None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:45.946491 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247092d-3e42-4839-a732-b7fffa3504ea None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:45.951987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72daa618-ba23-4f56-ac7f-4851ffa8fd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:45.952589 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72daa618-ba23-4f56-ac7f-4851ffa8fd9e admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:45.952878 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72daa618-ba23-4f56-ac7f-4851ffa8fd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:45.958147 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72daa618-ba23-4f56-ac7f-4851ffa8fd9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:45.958147 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72daa618-ba23-4f56-ac7f-4851ffa8fd9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:17:45.958147 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72daa618-ba23-4f56-ac7f-4851ffa8fd9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:45.958147 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72daa618-ba23-4f56-ac7f-4851ffa8fd9e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:45.968261 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1258/2541] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:17:45.972758 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63856b13-4211-44a1-9947-8e96d125f318 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:45.982040 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63856b13-4211-44a1-9947-8e96d125f318 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:45.987068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7789ed-0762-41f2-8158-3b1d3a98fb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:45.987668 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7789ed-0762-41f2-8158-3b1d3a98fb34 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:45.987949 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7789ed-0762-41f2-8158-3b1d3a98fb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:45.988228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7789ed-0762-41f2-8158-3b1d3a98fb34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:45.988862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7789ed-0762-41f2-8158-3b1d3a98fb34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:45.990782 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7789ed-0762-41f2-8158-3b1d3a98fb34 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:46.080001 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1284/2542] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:17:45 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:17:46.084489 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72daa618-ba23-4f56-ac7f-4851ffa8fd9e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:46.091409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:46.095022 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72daa618-ba23-4f56-ac7f-4851ffa8fd9e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:46.163010 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4e5e9-ffea-401a-81cc-b96c93b68375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:46.163642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4e5e9-ffea-401a-81cc-b96c93b68375 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:46.163933 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4e5e9-ffea-401a-81cc-b96c93b68375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:46.164215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4e5e9-ffea-401a-81cc-b96c93b68375 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:46.164872 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4e5e9-ffea-401a-81cc-b96c93b68375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:46.167209 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4e5e9-ffea-401a-81cc-b96c93b68375 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:46.260118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1259/2543] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:17:46 2022] POST /identity/v3/users => generated 283 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:17:46.266575 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7789ed-0762-41f2-8158-3b1d3a98fb34 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:46.277378 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7789ed-0762-41f2-8158-3b1d3a98fb34 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:46.282069 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fd22e-7628-463d-96b5-ad4297cb9a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:46.282665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fd22e-7628-463d-96b5-ad4297cb9a05 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:46.282942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fd22e-7628-463d-96b5-ad4297cb9a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:46.283216 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fd22e-7628-463d-96b5-ad4297cb9a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:46.283852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fd22e-7628-463d-96b5-ad4297cb9a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:46.285425 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fd22e-7628-463d-96b5-ad4297cb9a05 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:46.295490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1285/2544] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:17:46.299545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4e5e9-ffea-401a-81cc-b96c93b68375 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:46.310242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4e5e9-ffea-401a-81cc-b96c93b68375 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:46.315048 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70308f6-2185-414b-8853-9c5d5f458f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:46.315637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70308f6-2185-414b-8853-9c5d5f458f17 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:46.315923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70308f6-2185-414b-8853-9c5d5f458f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:46.320881 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70308f6-2185-414b-8853-9c5d5f458f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:46.320881 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70308f6-2185-414b-8853-9c5d5f458f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:46.320881 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70308f6-2185-414b-8853-9c5d5f458f17 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:46.330550 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1260/2545] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:17:46.334720 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fd22e-7628-463d-96b5-ad4297cb9a05 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:46.344751 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fd22e-7628-463d-96b5-ad4297cb9a05 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:46.349276 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22732839-dfa5-449d-bb11-cc0acb78e115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:46.351120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22732839-dfa5-449d-bb11-cc0acb78e115 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:46.351120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22732839-dfa5-449d-bb11-cc0acb78e115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:46.351120 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22732839-dfa5-449d-bb11-cc0acb78e115 admin admin] PATH_INFO: `/v3/projects/6d3fd39340054ec7b0b4b5f5b0b11632/users/bb006ba06b804329a7e5decc1a168d6a/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:46.365525 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22732839-dfa5-449d-bb11-cc0acb78e115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d3fd39340054ec7b0b4b5f5b0b11632, user_id=bb006ba06b804329a7e5decc1a168d6a, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:46.368260 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22732839-dfa5-449d-bb11-cc0acb78e115 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:46.450325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1286/2546] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:17:46 2022] PUT /identity/v3/projects/6d3fd39340054ec7b0b4b5f5b0b11632/users/bb006ba06b804329a7e5decc1a168d6a/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:46.460697 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c95a78-d34e-43a8-aed7-4b2b932f67dd None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:46.461051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c95a78-d34e-43a8-aed7-4b2b932f67dd None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:46.461333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c95a78-d34e-43a8-aed7-4b2b932f67dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:46.491286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-33c95a78-d34e-43a8-aed7-4b2b932f67dd None None] MFA Rules not processed for user `bb006ba06b804329a7e5decc1a168d6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:46.492529 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-33c95a78-d34e-43a8-aed7-4b2b932f67dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:46.559087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1261/2547] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:46.569095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22732839-dfa5-449d-bb11-cc0acb78e115 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:46.632921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22732839-dfa5-449d-bb11-cc0acb78e115 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:46.638372 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54496014-b0ef-47d7-b236-8b40bdc41573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:46.638982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54496014-b0ef-47d7-b236-8b40bdc41573 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:46.639269 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54496014-b0ef-47d7-b236-8b40bdc41573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:46.639573 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54496014-b0ef-47d7-b236-8b40bdc41573 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:46.640226 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54496014-b0ef-47d7-b236-8b40bdc41573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:46.642164 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54496014-b0ef-47d7-b236-8b40bdc41573 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:46.733210 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1287/2548] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:17:46 2022] POST /identity/v3/projects => generated 365 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:17:46.734370 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c95a78-d34e-43a8-aed7-4b2b932f67dd None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:46.745718 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c95a78-d34e-43a8-aed7-4b2b932f67dd None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:46.806760 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed44b44-364b-427c-8e6f-ae365b494cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:46.807378 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed44b44-364b-427c-8e6f-ae365b494cff admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:46.808304 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed44b44-364b-427c-8e6f-ae365b494cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:46.808645 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed44b44-364b-427c-8e6f-ae365b494cff admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:46.809359 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed44b44-364b-427c-8e6f-ae365b494cff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:46.811264 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed44b44-364b-427c-8e6f-ae365b494cff admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:46.889205 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1262/2549] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:17:46 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:17:46.893328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54496014-b0ef-47d7-b236-8b40bdc41573 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:46.903893 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54496014-b0ef-47d7-b236-8b40bdc41573 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:46.908447 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2374bcb-01a1-46a4-9940-268762f45a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:46.909040 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2374bcb-01a1-46a4-9940-268762f45a60 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:46.909318 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2374bcb-01a1-46a4-9940-268762f45a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:46.909590 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2374bcb-01a1-46a4-9940-268762f45a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:46.910285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2374bcb-01a1-46a4-9940-268762f45a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:46.911854 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2374bcb-01a1-46a4-9940-268762f45a60 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:46.922812 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1288/2550] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:17:46.927041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed44b44-364b-427c-8e6f-ae365b494cff admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:46.932286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:46.938190 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed44b44-364b-427c-8e6f-ae365b494cff admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:46.942869 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd0e6a-fc54-473f-afd0-04b1d10298f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:46.943515 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd0e6a-fc54-473f-afd0-04b1d10298f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:46.943800 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd0e6a-fc54-473f-afd0-04b1d10298f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:46.944169 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd0e6a-fc54-473f-afd0-04b1d10298f9 admin admin] PATH_INFO: `/v3/projects/dc181004957a4d879be21aa8a61ed9ac/users/f93f226883f545bd9d2c9d70f3c39ab3/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:46.959679 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd0e6a-fc54-473f-afd0-04b1d10298f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc181004957a4d879be21aa8a61ed9ac, user_id=f93f226883f545bd9d2c9d70f3c39ab3, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:46.962364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd0e6a-fc54-473f-afd0-04b1d10298f9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:47.002883 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1263/2551] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:17:46 2022] PUT /identity/v3/projects/dc181004957a4d879be21aa8a61ed9ac/users/f93f226883f545bd9d2c9d70f3c39ab3/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:47.006942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2374bcb-01a1-46a4-9940-268762f45a60 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:47.018666 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2374bcb-01a1-46a4-9940-268762f45a60 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:47.066869 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:47.077753 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ab4b6-6ffc-4e5c-857d-79f1f0e490d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:47.079944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ab4b6-6ffc-4e5c-857d-79f1f0e490d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:47.079944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ab4b6-6ffc-4e5c-857d-79f1f0e490d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:47.079944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ab4b6-6ffc-4e5c-857d-79f1f0e490d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:47.079944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ab4b6-6ffc-4e5c-857d-79f1f0e490d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:47.081144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ab4b6-6ffc-4e5c-857d-79f1f0e490d8 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:47.090764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1289/2552] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:17:47.094924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd0e6a-fc54-473f-afd0-04b1d10298f9 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:47.105477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd0e6a-fc54-473f-afd0-04b1d10298f9 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:47.110335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c40931-7d98-495f-b1b2-604817d509b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:47.110987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c40931-7d98-495f-b1b2-604817d509b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:47.111274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c40931-7d98-495f-b1b2-604817d509b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:47.111556 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c40931-7d98-495f-b1b2-604817d509b3 admin admin] PATH_INFO: `/v3/projects/dc181004957a4d879be21aa8a61ed9ac/users/f93f226883f545bd9d2c9d70f3c39ab3/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:47.113513 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c40931-7d98-495f-b1b2-604817d509b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc181004957a4d879be21aa8a61ed9ac, user_id=f93f226883f545bd9d2c9d70f3c39ab3, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:47.116191 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c40931-7d98-495f-b1b2-604817d509b3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:47.126969 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1264/2553] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:17:47 2022] PUT /identity/v3/projects/dc181004957a4d879be21aa8a61ed9ac/users/f93f226883f545bd9d2c9d70f3c39ab3/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:47.139418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078e1c2-3c33-4015-bf04-e116d5f8bde9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:47.139764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078e1c2-3c33-4015-bf04-e116d5f8bde9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:47.140053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078e1c2-3c33-4015-bf04-e116d5f8bde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:47.168833 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8078e1c2-3c33-4015-bf04-e116d5f8bde9 None None] MFA Rules not processed for user `f93f226883f545bd9d2c9d70f3c39ab3`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:47.170127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8078e1c2-3c33-4015-bf04-e116d5f8bde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:47.238364 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1290/2554] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:47.251231 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c40931-7d98-495f-b1b2-604817d509b3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:47.253612 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-41c40931-7d98-495f-b1b2-604817d509b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:47.313772 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c40931-7d98-495f-b1b2-604817d509b3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:47.317426 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42e065-63e0-4586-ae51-a04cf31bbdef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:47.318000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42e065-63e0-4586-ae51-a04cf31bbdef service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:47.318282 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42e065-63e0-4586-ae51-a04cf31bbdef service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:47.318559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42e065-63e0-4586-ae51-a04cf31bbdef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:47.327689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42e065-63e0-4586-ae51-a04cf31bbdef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:47.329119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42e065-63e0-4586-ae51-a04cf31bbdef service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:47.338769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1265/2555] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:17:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:17:48.305785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078e1c2-3c33-4015-bf04-e116d5f8bde9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:48.370112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078e1c2-3c33-4015-bf04-e116d5f8bde9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:48.375960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f39c9-d293-4c4a-a65d-9fa27f0ed018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:48.376377 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f39c9-d293-4c4a-a65d-9fa27f0ed018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:48.376664 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f39c9-d293-4c4a-a65d-9fa27f0ed018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:48.376943 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f39c9-d293-4c4a-a65d-9fa27f0ed018 admin admin] PATH_INFO: `/v3/users/bb006ba06b804329a7e5decc1a168d6a` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:48.378137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f39c9-d293-4c4a-a65d-9fa27f0ed018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb006ba06b804329a7e5decc1a168d6a)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:48.380116 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f39c9-d293-4c4a-a65d-9fa27f0ed018 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:48.536670 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-367f39c9-d293-4c4a-a65d-9fa27f0ed018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb006ba06b804329a7e5decc1a168d6a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:48.549926 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-367f39c9-d293-4c4a-a65d-9fa27f0ed018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb006ba06b804329a7e5decc1a168d6a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:48.557414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-367f39c9-d293-4c4a-a65d-9fa27f0ed018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb006ba06b804329a7e5decc1a168d6a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:48.559167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:48.647933 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-367f39c9-d293-4c4a-a65d-9fa27f0ed018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb006ba06b804329a7e5decc1a168d6a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:48.648552 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-367f39c9-d293-4c4a-a65d-9fa27f0ed018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb006ba06b804329a7e5decc1a168d6a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:48.656956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1291/2556] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:17:48 2022] DELETE /identity/v3/users/bb006ba06b804329a7e5decc1a168d6a => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:48.661715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29303aad-ad34-4995-8dc2-318cea953172 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:48.662222 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29303aad-ad34-4995-8dc2-318cea953172 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:48.662471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29303aad-ad34-4995-8dc2-318cea953172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:48.682963 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-29303aad-ad34-4995-8dc2-318cea953172 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:48.684229 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-29303aad-ad34-4995-8dc2-318cea953172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:48.747048 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1266/2557] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:48.756830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f39c9-d293-4c4a-a65d-9fa27f0ed018 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:48.758283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-367f39c9-d293-4c4a-a65d-9fa27f0ed018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:48.816623 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f39c9-d293-4c4a-a65d-9fa27f0ed018 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:48.825355 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfba7a-d3b1-4f1b-9e26-90126c51e50c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:48.826027 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfba7a-d3b1-4f1b-9e26-90126c51e50c service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:48.826309 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfba7a-d3b1-4f1b-9e26-90126c51e50c service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:48.826593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfba7a-d3b1-4f1b-9e26-90126c51e50c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:48.837246 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfba7a-d3b1-4f1b-9e26-90126c51e50c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:48.838716 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfba7a-d3b1-4f1b-9e26-90126c51e50c service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:48.849370 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1292/2558] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:17:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:17:49.411396 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29303aad-ad34-4995-8dc2-318cea953172 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:49.413022 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-29303aad-ad34-4995-8dc2-318cea953172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:49.431698 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29303aad-ad34-4995-8dc2-318cea953172 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:49.437769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4a362-66a9-4a7c-966d-51565f532792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:49.438643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4a362-66a9-4a7c-966d-51565f532792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:49.439712 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4a362-66a9-4a7c-966d-51565f532792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:49.439912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4a362-66a9-4a7c-966d-51565f532792 admin admin] PATH_INFO: `/v3/projects/6d3fd39340054ec7b0b4b5f5b0b11632` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:49.441157 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4a362-66a9-4a7c-966d-51565f532792 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d3fd39340054ec7b0b4b5f5b0b11632)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:49.444002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4a362-66a9-4a7c-966d-51565f532792 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:49.550113 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-55d4a362-66a9-4a7c-966d-51565f532792 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d3fd39340054ec7b0b4b5f5b0b11632', 'request_id': 'req-55d4a362-66a9-4a7c-966d-51565f532792'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:49.602434 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-55d4a362-66a9-4a7c-966d-51565f532792 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d3fd39340054ec7b0b4b5f5b0b11632', 'request_id': 'req-55d4a362-66a9-4a7c-966d-51565f532792'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:49.608242 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:49.609724 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-55d4a362-66a9-4a7c-966d-51565f532792 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d3fd39340054ec7b0b4b5f5b0b11632', 'request_id': 'req-55d4a362-66a9-4a7c-966d-51565f532792'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:49.621487 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-55d4a362-66a9-4a7c-966d-51565f532792 admin admin] The token cache is being invalidate because project 6d3fd39340054ec7b0b4b5f5b0b11632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:17:49.621804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-55d4a362-66a9-4a7c-966d-51565f532792 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:49.622489 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-55d4a362-66a9-4a7c-966d-51565f532792 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:49.623982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1267/2559] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:17:49 2022] DELETE /identity/v3/projects/6d3fd39340054ec7b0b4b5f5b0b11632 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:49.628057 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfba7a-d3b1-4f1b-9e26-90126c51e50c service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:49.638818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfba7a-d3b1-4f1b-9e26-90126c51e50c service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:49.698655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78e2ef-f89f-4838-8389-11ce6d1e9283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:49.699138 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78e2ef-f89f-4838-8389-11ce6d1e9283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:49.699593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78e2ef-f89f-4838-8389-11ce6d1e9283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:49.699905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78e2ef-f89f-4838-8389-11ce6d1e9283 admin admin] PATH_INFO: `/v3/users/f93f226883f545bd9d2c9d70f3c39ab3` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:49.701051 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca78e2ef-f89f-4838-8389-11ce6d1e9283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f93f226883f545bd9d2c9d70f3c39ab3)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:49.702993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca78e2ef-f89f-4838-8389-11ce6d1e9283 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:49.834620 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ca78e2ef-f89f-4838-8389-11ce6d1e9283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f93f226883f545bd9d2c9d70f3c39ab3'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:49.847798 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ca78e2ef-f89f-4838-8389-11ce6d1e9283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f93f226883f545bd9d2c9d70f3c39ab3'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:49.854717 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ca78e2ef-f89f-4838-8389-11ce6d1e9283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f93f226883f545bd9d2c9d70f3c39ab3'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:49.905357 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ca78e2ef-f89f-4838-8389-11ce6d1e9283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f93f226883f545bd9d2c9d70f3c39ab3'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:49.906021 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ca78e2ef-f89f-4838-8389-11ce6d1e9283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f93f226883f545bd9d2c9d70f3c39ab3'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:49.912660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1293/2560] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:17:49 2022] DELETE /identity/v3/users/f93f226883f545bd9d2c9d70f3c39ab3 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:50.307085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4a362-66a9-4a7c-966d-51565f532792 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:50.309625 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-55d4a362-66a9-4a7c-966d-51565f532792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:50.323779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4a362-66a9-4a7c-966d-51565f532792 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:50.385754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e31572-7cb8-4c20-ae61-c48c3c66b1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:50.386428 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e31572-7cb8-4c20-ae61-c48c3c66b1a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:50.386712 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e31572-7cb8-4c20-ae61-c48c3c66b1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:50.386989 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e31572-7cb8-4c20-ae61-c48c3c66b1a0 admin admin] PATH_INFO: `/v3/projects/dc181004957a4d879be21aa8a61ed9ac` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:50.388143 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e31572-7cb8-4c20-ae61-c48c3c66b1a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc181004957a4d879be21aa8a61ed9ac)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:50.390097 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e31572-7cb8-4c20-ae61-c48c3c66b1a0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:50.495879 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-21e31572-7cb8-4c20-ae61-c48c3c66b1a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc181004957a4d879be21aa8a61ed9ac', 'request_id': 'req-21e31572-7cb8-4c20-ae61-c48c3c66b1a0'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:50.567875 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-21e31572-7cb8-4c20-ae61-c48c3c66b1a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc181004957a4d879be21aa8a61ed9ac', 'request_id': 'req-21e31572-7cb8-4c20-ae61-c48c3c66b1a0'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:50.573779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-21e31572-7cb8-4c20-ae61-c48c3c66b1a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc181004957a4d879be21aa8a61ed9ac', 'request_id': 'req-21e31572-7cb8-4c20-ae61-c48c3c66b1a0'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:50.585518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-21e31572-7cb8-4c20-ae61-c48c3c66b1a0 admin admin] The token cache is being invalidate because project dc181004957a4d879be21aa8a61ed9ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:17:50.585813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-21e31572-7cb8-4c20-ae61-c48c3c66b1a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:50.586496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-21e31572-7cb8-4c20-ae61-c48c3c66b1a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:50.589412 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1268/2561] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:17:50 2022] DELETE /identity/v3/projects/dc181004957a4d879be21aa8a61ed9ac => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:50.593451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2dd14-971d-4bf2-98da-0923d374935a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:50.593739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2dd14-971d-4bf2-98da-0923d374935a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:50.594038 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2dd14-971d-4bf2-98da-0923d374935a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:50.614141 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2bd2dd14-971d-4bf2-98da-0923d374935a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:50.617813 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2bd2dd14-971d-4bf2-98da-0923d374935a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:50.623660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:50.672916 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1294/2562] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:50.681088 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defbc3b-8021-48c0-94c3-13d141798677 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:50.681379 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defbc3b-8021-48c0-94c3-13d141798677 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:50.681664 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defbc3b-8021-48c0-94c3-13d141798677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:50.701552 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-5defbc3b-8021-48c0-94c3-13d141798677 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:50.702817 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5defbc3b-8021-48c0-94c3-13d141798677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:50.710769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1269/2563] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:17:50.715471 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2dd14-971d-4bf2-98da-0923d374935a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:50.725040 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2dd14-971d-4bf2-98da-0923d374935a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:50.730093 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c7889-e9dd-434e-8969-52dd4da981d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:50.730694 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c7889-e9dd-434e-8969-52dd4da981d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:50.730972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c7889-e9dd-434e-8969-52dd4da981d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:50.731249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c7889-e9dd-434e-8969-52dd4da981d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:50.731911 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c7889-e9dd-434e-8969-52dd4da981d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:17:50.732212 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c7889-e9dd-434e-8969-52dd4da981d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:50.733792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c7889-e9dd-434e-8969-52dd4da981d2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:50.745064 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1295/2564] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 12:17:50.749297 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5defbc3b-8021-48c0-94c3-13d141798677 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:50.758697 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5defbc3b-8021-48c0-94c3-13d141798677 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:50.763369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae623ea3-c1bb-4cf3-8dc5-282389268a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:50.763964 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae623ea3-c1bb-4cf3-8dc5-282389268a6e admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:50.764246 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae623ea3-c1bb-4cf3-8dc5-282389268a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:50.764544 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae623ea3-c1bb-4cf3-8dc5-282389268a6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:50.765184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae623ea3-c1bb-4cf3-8dc5-282389268a6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:50.767121 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae623ea3-c1bb-4cf3-8dc5-282389268a6e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:50.861767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1270/2565] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:17:50 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:17:50.866174 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c7889-e9dd-434e-8969-52dd4da981d2 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:50.876376 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c7889-e9dd-434e-8969-52dd4da981d2 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:50.934789 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458dd55f-ec86-4a65-b88c-f08fd46273fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:50.935398 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dd55f-ec86-4a65-b88c-f08fd46273fd admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:50.935681 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dd55f-ec86-4a65-b88c-f08fd46273fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:50.935959 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dd55f-ec86-4a65-b88c-f08fd46273fd admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:50.936599 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458dd55f-ec86-4a65-b88c-f08fd46273fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:50.938621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458dd55f-ec86-4a65-b88c-f08fd46273fd admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:51.007903 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1296/2566] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:17:50 2022] POST /identity/v3/users => generated 294 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:17:51.012673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae623ea3-c1bb-4cf3-8dc5-282389268a6e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:51.023431 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae623ea3-c1bb-4cf3-8dc5-282389268a6e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:51.028793 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efb31b-fc21-45de-84cd-0ce7b05105d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:51.029390 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efb31b-fc21-45de-84cd-0ce7b05105d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:51.029673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efb31b-fc21-45de-84cd-0ce7b05105d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:51.030087 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efb31b-fc21-45de-84cd-0ce7b05105d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:51.030730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efb31b-fc21-45de-84cd-0ce7b05105d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:51.032320 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efb31b-fc21-45de-84cd-0ce7b05105d4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:51.042256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1271/2567] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:17:51.046266 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458dd55f-ec86-4a65-b88c-f08fd46273fd admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:51.056197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458dd55f-ec86-4a65-b88c-f08fd46273fd admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:51.060821 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd1842-e811-4ed4-8f76-397d9132537f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:51.061409 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd1842-e811-4ed4-8f76-397d9132537f admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:51.061691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd1842-e811-4ed4-8f76-397d9132537f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:51.062068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd1842-e811-4ed4-8f76-397d9132537f admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:51.062700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd1842-e811-4ed4-8f76-397d9132537f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:51.064278 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd1842-e811-4ed4-8f76-397d9132537f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:51.075151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1297/2568] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:17:51.079285 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efb31b-fc21-45de-84cd-0ce7b05105d4 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:51.090125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efb31b-fc21-45de-84cd-0ce7b05105d4 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:51.094780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ab2e1-2da9-458e-a222-38a84e302931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:51.095427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ab2e1-2da9-458e-a222-38a84e302931 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:51.095716 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ab2e1-2da9-458e-a222-38a84e302931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:51.095997 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ab2e1-2da9-458e-a222-38a84e302931 admin admin] PATH_INFO: `/v3/projects/7392f80fb6d64b15bfab3e05a9152375/users/b5de007887034c7fbe4449424ce3f0d6/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:51.110405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ab2e1-2da9-458e-a222-38a84e302931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7392f80fb6d64b15bfab3e05a9152375, user_id=b5de007887034c7fbe4449424ce3f0d6, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:51.113057 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ab2e1-2da9-458e-a222-38a84e302931 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:51.184981 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1272/2569] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:17:51 2022] PUT /identity/v3/projects/7392f80fb6d64b15bfab3e05a9152375/users/b5de007887034c7fbe4449424ce3f0d6/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:17:51.195391 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c170775-009b-4bd0-ba0c-cbb62b59b2b4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:51.201084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c170775-009b-4bd0-ba0c-cbb62b59b2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:51.201084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c170775-009b-4bd0-ba0c-cbb62b59b2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:51.224481 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8c170775-009b-4bd0-ba0c-cbb62b59b2b4 None None] MFA Rules not processed for user `b5de007887034c7fbe4449424ce3f0d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:51.225695 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8c170775-009b-4bd0-ba0c-cbb62b59b2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:51.295300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1298/2570] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:51.300054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ab2e1-2da9-458e-a222-38a84e302931 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:51.320156 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:51.362875 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ab2e1-2da9-458e-a222-38a84e302931 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:51.367635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c69a6-355c-456b-8767-1ce2b1f509c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:51.368240 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c69a6-355c-456b-8767-1ce2b1f509c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:51.368526 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c69a6-355c-456b-8767-1ce2b1f509c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:51.368813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c69a6-355c-456b-8767-1ce2b1f509c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:51.369455 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c69a6-355c-456b-8767-1ce2b1f509c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:51.371908 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c69a6-355c-456b-8767-1ce2b1f509c3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:51.466067 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1273/2571] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:17:51 2022] POST /identity/v3/projects => generated 387 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:17:51.470193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c170775-009b-4bd0-ba0c-cbb62b59b2b4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:51.481093 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c170775-009b-4bd0-ba0c-cbb62b59b2b4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:51.486119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:51.538421 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb8302-56c0-4509-9fee-fdd47824281b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:51.539041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb8302-56c0-4509-9fee-fdd47824281b admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:51.539325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb8302-56c0-4509-9fee-fdd47824281b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:51.539605 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb8302-56c0-4509-9fee-fdd47824281b admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:51.540241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb8302-56c0-4509-9fee-fdd47824281b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:51.542182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb8302-56c0-4509-9fee-fdd47824281b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:51.657022 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1299/2572] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:17:51 2022] POST /identity/v3/users => generated 300 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:17:51.661322 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c69a6-355c-456b-8767-1ce2b1f509c3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:51.672193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c69a6-355c-456b-8767-1ce2b1f509c3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:51.676856 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763a8a0-fb86-4c77-abf1-1fdcd8c15f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:51.680129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763a8a0-fb86-4c77-abf1-1fdcd8c15f2b admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:51.680129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763a8a0-fb86-4c77-abf1-1fdcd8c15f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:51.680129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763a8a0-fb86-4c77-abf1-1fdcd8c15f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:51.680129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763a8a0-fb86-4c77-abf1-1fdcd8c15f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:51.680517 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763a8a0-fb86-4c77-abf1-1fdcd8c15f2b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:51.691866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1274/2573] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:17:51.696122 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb8302-56c0-4509-9fee-fdd47824281b admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:51.707046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb8302-56c0-4509-9fee-fdd47824281b admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:51.711728 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a6435-55fe-42c3-9615-eb2bc574e7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:51.712376 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a6435-55fe-42c3-9615-eb2bc574e7df admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:51.712660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a6435-55fe-42c3-9615-eb2bc574e7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:51.712936 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a6435-55fe-42c3-9615-eb2bc574e7df admin admin] PATH_INFO: `/v3/projects/6ee75b72712549c9b0ca9a45c0322f70/users/de246f67019c407eb0fe0967fa2f387a/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:51.727491 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a6435-55fe-42c3-9615-eb2bc574e7df admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee75b72712549c9b0ca9a45c0322f70, user_id=de246f67019c407eb0fe0967fa2f387a, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:51.730716 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a6435-55fe-42c3-9615-eb2bc574e7df admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:51.814285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1300/2574] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:17:51 2022] PUT /identity/v3/projects/6ee75b72712549c9b0ca9a45c0322f70/users/de246f67019c407eb0fe0967fa2f387a/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:17:51.818400 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763a8a0-fb86-4c77-abf1-1fdcd8c15f2b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:51.832711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763a8a0-fb86-4c77-abf1-1fdcd8c15f2b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:51.889836 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27527159-6d72-4e98-8819-ae771d60aec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:51.890397 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27527159-6d72-4e98-8819-ae771d60aec7 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:51.890686 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27527159-6d72-4e98-8819-ae771d60aec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:51.890998 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27527159-6d72-4e98-8819-ae771d60aec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:51.891680 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27527159-6d72-4e98-8819-ae771d60aec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:51.893310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27527159-6d72-4e98-8819-ae771d60aec7 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:51.903770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1275/2575] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:17:51.907976 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a6435-55fe-42c3-9615-eb2bc574e7df admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:51.919645 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a6435-55fe-42c3-9615-eb2bc574e7df admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:51.923714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb058e-3a81-4bdf-a00a-e41afa317687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:51.924359 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb058e-3a81-4bdf-a00a-e41afa317687 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:51.924664 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb058e-3a81-4bdf-a00a-e41afa317687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:51.924942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb058e-3a81-4bdf-a00a-e41afa317687 admin admin] PATH_INFO: `/v3/projects/6ee75b72712549c9b0ca9a45c0322f70/users/de246f67019c407eb0fe0967fa2f387a/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:51.926910 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb058e-3a81-4bdf-a00a-e41afa317687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee75b72712549c9b0ca9a45c0322f70, user_id=de246f67019c407eb0fe0967fa2f387a, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:51.932200 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb058e-3a81-4bdf-a00a-e41afa317687 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:51.940345 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1301/2576] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:17:51 2022] PUT /identity/v3/projects/6ee75b72712549c9b0ca9a45c0322f70/users/de246f67019c407eb0fe0967fa2f387a/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:51.951771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f27d39-033c-47bd-9b99-e7d14d01d56d None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:51.952058 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f27d39-033c-47bd-9b99-e7d14d01d56d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:51.952338 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f27d39-033c-47bd-9b99-e7d14d01d56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:51.986206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e9f27d39-033c-47bd-9b99-e7d14d01d56d None None] MFA Rules not processed for user `de246f67019c407eb0fe0967fa2f387a`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:51.987433 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e9f27d39-033c-47bd-9b99-e7d14d01d56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:52.057319 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1276/2577] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:51 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:52.069504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb058e-3a81-4bdf-a00a-e41afa317687 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:52.071269 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2cfb058e-3a81-4bdf-a00a-e41afa317687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:52.133257 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb058e-3a81-4bdf-a00a-e41afa317687 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:52.137146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9dc4c5-b811-4c9b-bcfa-260fe42f90fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:52.137764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9dc4c5-b811-4c9b-bcfa-260fe42f90fd service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:52.138070 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9dc4c5-b811-4c9b-bcfa-260fe42f90fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:52.138418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9dc4c5-b811-4c9b-bcfa-260fe42f90fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:52.148915 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9dc4c5-b811-4c9b-bcfa-260fe42f90fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:52.150473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9dc4c5-b811-4c9b-bcfa-260fe42f90fd service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:52.161616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1302/2578] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:17:52 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:17:53.212402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f27d39-033c-47bd-9b99-e7d14d01d56d None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:53.225118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:53.275053 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f27d39-033c-47bd-9b99-e7d14d01d56d None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:53.279945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb70f2-6c1d-4741-ada1-e089fc7d5de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:53.280589 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb70f2-6c1d-4741-ada1-e089fc7d5de3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:53.280875 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb70f2-6c1d-4741-ada1-e089fc7d5de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:53.281151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb70f2-6c1d-4741-ada1-e089fc7d5de3 admin admin] PATH_INFO: `/v3/users/b5de007887034c7fbe4449424ce3f0d6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:53.282327 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb70f2-6c1d-4741-ada1-e089fc7d5de3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5de007887034c7fbe4449424ce3f0d6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:53.284255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb70f2-6c1d-4741-ada1-e089fc7d5de3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:53.491462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-46cb70f2-6c1d-4741-ada1-e089fc7d5de3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5de007887034c7fbe4449424ce3f0d6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:53.504127 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-46cb70f2-6c1d-4741-ada1-e089fc7d5de3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5de007887034c7fbe4449424ce3f0d6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:53.511131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-46cb70f2-6c1d-4741-ada1-e089fc7d5de3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5de007887034c7fbe4449424ce3f0d6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:53.590730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-46cb70f2-6c1d-4741-ada1-e089fc7d5de3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5de007887034c7fbe4449424ce3f0d6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:53.591341 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-46cb70f2-6c1d-4741-ada1-e089fc7d5de3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5de007887034c7fbe4449424ce3f0d6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:53.597517 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1277/2579] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:17:53 2022] DELETE /identity/v3/users/b5de007887034c7fbe4449424ce3f0d6 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:53.602090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f852d-dea2-4ebf-9a57-a9d5f2867b3b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:53.602612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f852d-dea2-4ebf-9a57-a9d5f2867b3b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:53.602612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f852d-dea2-4ebf-9a57-a9d5f2867b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:53.622706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-828f852d-dea2-4ebf-9a57-a9d5f2867b3b None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:53.623936 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-828f852d-dea2-4ebf-9a57-a9d5f2867b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:53.680467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1303/2580] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:53.690285 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb70f2-6c1d-4741-ada1-e089fc7d5de3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:53.691692 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-46cb70f2-6c1d-4741-ada1-e089fc7d5de3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:53.748900 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb70f2-6c1d-4741-ada1-e089fc7d5de3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:53.752411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6abea-3f44-4b61-bb57-398898029b71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:53.753007 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6abea-3f44-4b61-bb57-398898029b71 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:53.753288 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6abea-3f44-4b61-bb57-398898029b71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:53.753563 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6abea-3f44-4b61-bb57-398898029b71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:53.762583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6abea-3f44-4b61-bb57-398898029b71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:53.764092 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6abea-3f44-4b61-bb57-398898029b71 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:53.773829 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1278/2581] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:17:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:17:54.255408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f852d-dea2-4ebf-9a57-a9d5f2867b3b None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:54.258179 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-828f852d-dea2-4ebf-9a57-a9d5f2867b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:54.265637 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:54.270067 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f852d-dea2-4ebf-9a57-a9d5f2867b3b None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:54.274981 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e8313-eeac-429d-bfd0-6f91c654cb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:54.275661 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e8313-eeac-429d-bfd0-6f91c654cb04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:54.275924 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e8313-eeac-429d-bfd0-6f91c654cb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:54.276241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e8313-eeac-429d-bfd0-6f91c654cb04 admin admin] PATH_INFO: `/v3/projects/7392f80fb6d64b15bfab3e05a9152375` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:54.277994 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e8313-eeac-429d-bfd0-6f91c654cb04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7392f80fb6d64b15bfab3e05a9152375)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:54.279904 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e8313-eeac-429d-bfd0-6f91c654cb04 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:54.408309 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e06e8313-eeac-429d-bfd0-6f91c654cb04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7392f80fb6d64b15bfab3e05a9152375', 'request_id': 'req-e06e8313-eeac-429d-bfd0-6f91c654cb04'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:54.465396 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e06e8313-eeac-429d-bfd0-6f91c654cb04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7392f80fb6d64b15bfab3e05a9152375', 'request_id': 'req-e06e8313-eeac-429d-bfd0-6f91c654cb04'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:54.472321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e06e8313-eeac-429d-bfd0-6f91c654cb04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7392f80fb6d64b15bfab3e05a9152375', 'request_id': 'req-e06e8313-eeac-429d-bfd0-6f91c654cb04'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:54.483885 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e06e8313-eeac-429d-bfd0-6f91c654cb04 admin admin] The token cache is being invalidate because project 7392f80fb6d64b15bfab3e05a9152375 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:17:54.484176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e06e8313-eeac-429d-bfd0-6f91c654cb04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:54.484831 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e06e8313-eeac-429d-bfd0-6f91c654cb04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:54.486482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1304/2582] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:17:54 2022] DELETE /identity/v3/projects/7392f80fb6d64b15bfab3e05a9152375 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:54.490531 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6abea-3f44-4b61-bb57-398898029b71 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:54.501003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6abea-3f44-4b61-bb57-398898029b71 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:54.534048 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:54.558494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2436d-a914-4df4-9364-5871508b54a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:54.559154 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2436d-a914-4df4-9364-5871508b54a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:54.559437 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2436d-a914-4df4-9364-5871508b54a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:54.559712 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2436d-a914-4df4-9364-5871508b54a5 admin admin] PATH_INFO: `/v3/users/de246f67019c407eb0fe0967fa2f387a` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:54.560846 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d2436d-a914-4df4-9364-5871508b54a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de246f67019c407eb0fe0967fa2f387a)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:54.562798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d2436d-a914-4df4-9364-5871508b54a5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:54.694899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-09d2436d-a914-4df4-9364-5871508b54a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de246f67019c407eb0fe0967fa2f387a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:54.708337 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-09d2436d-a914-4df4-9364-5871508b54a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de246f67019c407eb0fe0967fa2f387a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:54.715730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-09d2436d-a914-4df4-9364-5871508b54a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de246f67019c407eb0fe0967fa2f387a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:54.785403 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-09d2436d-a914-4df4-9364-5871508b54a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de246f67019c407eb0fe0967fa2f387a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:54.786074 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-09d2436d-a914-4df4-9364-5871508b54a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de246f67019c407eb0fe0967fa2f387a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:54.793986 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1279/2583] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:17:54 2022] DELETE /identity/v3/users/de246f67019c407eb0fe0967fa2f387a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:55.221709 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e8313-eeac-429d-bfd0-6f91c654cb04 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:55.223370 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e06e8313-eeac-429d-bfd0-6f91c654cb04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:55.237345 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e8313-eeac-429d-bfd0-6f91c654cb04 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:55.299938 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d4988-003d-49f9-b505-339c30b58124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:55.300514 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d4988-003d-49f9-b505-339c30b58124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:55.300796 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d4988-003d-49f9-b505-339c30b58124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:55.301146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d4988-003d-49f9-b505-339c30b58124 admin admin] PATH_INFO: `/v3/projects/6ee75b72712549c9b0ca9a45c0322f70` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:55.302375 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d4988-003d-49f9-b505-339c30b58124 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ee75b72712549c9b0ca9a45c0322f70)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:55.304424 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d4988-003d-49f9-b505-339c30b58124 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:55.422137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-da7d4988-003d-49f9-b505-339c30b58124 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ee75b72712549c9b0ca9a45c0322f70', 'request_id': 'req-da7d4988-003d-49f9-b505-339c30b58124'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:55.505248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-da7d4988-003d-49f9-b505-339c30b58124 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ee75b72712549c9b0ca9a45c0322f70', 'request_id': 'req-da7d4988-003d-49f9-b505-339c30b58124'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:55.512124 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-da7d4988-003d-49f9-b505-339c30b58124 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ee75b72712549c9b0ca9a45c0322f70', 'request_id': 'req-da7d4988-003d-49f9-b505-339c30b58124'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:55.523262 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-da7d4988-003d-49f9-b505-339c30b58124 admin admin] The token cache is being invalidate because project 6ee75b72712549c9b0ca9a45c0322f70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:17:55.523549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-da7d4988-003d-49f9-b505-339c30b58124 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:55.524193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-da7d4988-003d-49f9-b505-339c30b58124 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:17:55.525655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1305/2584] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:17:55 2022] DELETE /identity/v3/projects/6ee75b72712549c9b0ca9a45c0322f70 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:55.530734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20c4a3-6eb8-48ec-bb83-994d7ee4e110 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:55.531226 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20c4a3-6eb8-48ec-bb83-994d7ee4e110 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:55.531335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20c4a3-6eb8-48ec-bb83-994d7ee4e110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:55.551315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-0a20c4a3-6eb8-48ec-bb83-994d7ee4e110 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:55.552464 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0a20c4a3-6eb8-48ec-bb83-994d7ee4e110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:55.611710 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1280/2585] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:55.620617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a08664-8d2a-4285-842f-08928d81961a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:55.620920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a08664-8d2a-4285-842f-08928d81961a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:55.621200 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a08664-8d2a-4285-842f-08928d81961a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:55.640593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-52a08664-8d2a-4285-842f-08928d81961a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:55.641796 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-52a08664-8d2a-4285-842f-08928d81961a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:55.650007 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1306/2586] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:55.654511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20c4a3-6eb8-48ec-bb83-994d7ee4e110 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:55.664210 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20c4a3-6eb8-48ec-bb83-994d7ee4e110 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:55.669346 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5f1d6-2295-470f-a529-f3e37359d419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:55.669975 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5f1d6-2295-470f-a529-f3e37359d419 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:55.670264 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5f1d6-2295-470f-a529-f3e37359d419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:55.670543 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5f1d6-2295-470f-a529-f3e37359d419 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:55.671195 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5f1d6-2295-470f-a529-f3e37359d419 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:17:55.671503 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5f1d6-2295-470f-a529-f3e37359d419 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:55.673105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5f1d6-2295-470f-a529-f3e37359d419 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:55.685184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1281/2587] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:17:55.689573 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a08664-8d2a-4285-842f-08928d81961a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:55.698888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a08664-8d2a-4285-842f-08928d81961a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:55.703569 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda798c-b293-4051-a7ba-9f3a9ae10980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:55.704161 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda798c-b293-4051-a7ba-9f3a9ae10980 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:55.704447 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda798c-b293-4051-a7ba-9f3a9ae10980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:55.704726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda798c-b293-4051-a7ba-9f3a9ae10980 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:55.705357 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda798c-b293-4051-a7ba-9f3a9ae10980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:55.707296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda798c-b293-4051-a7ba-9f3a9ae10980 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:55.716723 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:55.792562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1307/2588] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:17:55 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:17:55.797020 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5f1d6-2295-470f-a529-f3e37359d419 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:55.808117 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5f1d6-2295-470f-a529-f3e37359d419 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:55.849664 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:55.866403 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4739ea7-6601-44b5-8d62-7fa96d7662c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:55.867004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4739ea7-6601-44b5-8d62-7fa96d7662c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:55.867286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4739ea7-6601-44b5-8d62-7fa96d7662c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:55.867560 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4739ea7-6601-44b5-8d62-7fa96d7662c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:55.868184 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4739ea7-6601-44b5-8d62-7fa96d7662c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:55.870220 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4739ea7-6601-44b5-8d62-7fa96d7662c3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:55.973999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1282/2589] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:17:55 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:17:55.978227 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda798c-b293-4051-a7ba-9f3a9ae10980 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:55.989171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda798c-b293-4051-a7ba-9f3a9ae10980 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:55.993832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a8a55-0670-486b-9b89-4204da4e9e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:55.994455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a8a55-0670-486b-9b89-4204da4e9e49 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:55.994736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a8a55-0670-486b-9b89-4204da4e9e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:55.995189 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a8a55-0670-486b-9b89-4204da4e9e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:55.995763 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a8a55-0670-486b-9b89-4204da4e9e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:55.997342 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a8a55-0670-486b-9b89-4204da4e9e49 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:56.007231 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1308/2590] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:17:56.011154 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4739ea7-6601-44b5-8d62-7fa96d7662c3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:56.020973 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4739ea7-6601-44b5-8d62-7fa96d7662c3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:56.027175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f5806c-fcfe-4040-ac17-b294fd2a7e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:56.027776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5806c-fcfe-4040-ac17-b294fd2a7e9e admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:56.028088 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5806c-fcfe-4040-ac17-b294fd2a7e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:56.028365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5806c-fcfe-4040-ac17-b294fd2a7e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:56.029062 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5806c-fcfe-4040-ac17-b294fd2a7e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:56.030859 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5806c-fcfe-4040-ac17-b294fd2a7e9e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:56.041650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1283/2591] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:17:56.046198 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a8a55-0670-486b-9b89-4204da4e9e49 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:56.055836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a8a55-0670-486b-9b89-4204da4e9e49 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:56.060446 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f85db-c2dd-4d9a-be3d-2ffe824132be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:56.061041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f85db-c2dd-4d9a-be3d-2ffe824132be admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:56.061304 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f85db-c2dd-4d9a-be3d-2ffe824132be admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:56.061575 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f85db-c2dd-4d9a-be3d-2ffe824132be admin admin] PATH_INFO: `/v3/projects/0bfb0b4014b347af878e62d7be4d80e5/users/a474f3be7d514b39ada1a487ed76d957/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:56.076376 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0f85db-c2dd-4d9a-be3d-2ffe824132be admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bfb0b4014b347af878e62d7be4d80e5, user_id=a474f3be7d514b39ada1a487ed76d957, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:56.079015 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0f85db-c2dd-4d9a-be3d-2ffe824132be admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:56.180223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1309/2592] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:17:56 2022] PUT /identity/v3/projects/0bfb0b4014b347af878e62d7be4d80e5/users/a474f3be7d514b39ada1a487ed76d957/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:56.190611 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c525445-a8fb-4b86-8189-f31e0bd49a08 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:56.190907 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c525445-a8fb-4b86-8189-f31e0bd49a08 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:56.191603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c525445-a8fb-4b86-8189-f31e0bd49a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:56.221509 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4c525445-a8fb-4b86-8189-f31e0bd49a08 None None] MFA Rules not processed for user `a474f3be7d514b39ada1a487ed76d957`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:56.222945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4c525445-a8fb-4b86-8189-f31e0bd49a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:56.290373 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1284/2593] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:56.294881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f85db-c2dd-4d9a-be3d-2ffe824132be admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:56.359426 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f85db-c2dd-4d9a-be3d-2ffe824132be admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:56.364503 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec021e73-e148-46d0-b396-ef650affbace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:56.365147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec021e73-e148-46d0-b396-ef650affbace admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:56.365443 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec021e73-e148-46d0-b396-ef650affbace admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:56.365755 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec021e73-e148-46d0-b396-ef650affbace admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:56.366527 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec021e73-e148-46d0-b396-ef650affbace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:56.368775 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec021e73-e148-46d0-b396-ef650affbace admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:56.598565 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1310/2594] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:17:56 2022] POST /identity/v3/projects => generated 385 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:17:56.602566 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c525445-a8fb-4b86-8189-f31e0bd49a08 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:56.613166 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c525445-a8fb-4b86-8189-f31e0bd49a08 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:56.672392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c205f2-db27-47cb-9973-b4dd29073df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:56.673121 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c205f2-db27-47cb-9973-b4dd29073df2 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:56.673473 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c205f2-db27-47cb-9973-b4dd29073df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:56.673651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c205f2-db27-47cb-9973-b4dd29073df2 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:56.674328 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c205f2-db27-47cb-9973-b4dd29073df2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:56.676259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c205f2-db27-47cb-9973-b4dd29073df2 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:56.768978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1285/2595] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:17:56 2022] POST /identity/v3/users => generated 299 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:17:56.773795 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec021e73-e148-46d0-b396-ef650affbace admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:56.783647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec021e73-e148-46d0-b396-ef650affbace admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:56.788371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160b826-7e0b-4128-a60a-ef4b6634413c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:56.789050 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160b826-7e0b-4128-a60a-ef4b6634413c admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:56.789770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160b826-7e0b-4128-a60a-ef4b6634413c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:56.789770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160b826-7e0b-4128-a60a-ef4b6634413c admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:56.790248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d160b826-7e0b-4128-a60a-ef4b6634413c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:56.791876 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d160b826-7e0b-4128-a60a-ef4b6634413c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:56.802751 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1311/2596] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:17:56.807185 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c205f2-db27-47cb-9973-b4dd29073df2 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:56.818000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c205f2-db27-47cb-9973-b4dd29073df2 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:56.834061 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795733b-c77c-420d-9d15-13d8d121e737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:56.834061 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795733b-c77c-420d-9d15-13d8d121e737 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:56.834061 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795733b-c77c-420d-9d15-13d8d121e737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:56.834061 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795733b-c77c-420d-9d15-13d8d121e737 admin admin] PATH_INFO: `/v3/projects/24b58344b7da4672b31f1c0891afeac7/users/a7bd270828bc424b91bebaf1c7a48e62/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:56.846657 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795733b-c77c-420d-9d15-13d8d121e737 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24b58344b7da4672b31f1c0891afeac7, user_id=a7bd270828bc424b91bebaf1c7a48e62, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:56.849388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795733b-c77c-420d-9d15-13d8d121e737 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:56.975691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1286/2597] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:17:56 2022] PUT /identity/v3/projects/24b58344b7da4672b31f1c0891afeac7/users/a7bd270828bc424b91bebaf1c7a48e62/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:17:56.982129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160b826-7e0b-4128-a60a-ef4b6634413c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:56.999949 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160b826-7e0b-4128-a60a-ef4b6634413c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:57.058879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694c242-c02b-4bcf-9c84-e2ea34b93d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:57.059483 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694c242-c02b-4bcf-9c84-e2ea34b93d04 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:57.059761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694c242-c02b-4bcf-9c84-e2ea34b93d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:57.060034 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694c242-c02b-4bcf-9c84-e2ea34b93d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:57.060660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8694c242-c02b-4bcf-9c84-e2ea34b93d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:57.062277 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8694c242-c02b-4bcf-9c84-e2ea34b93d04 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:57.071934 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1312/2598] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:17:57.076163 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795733b-c77c-420d-9d15-13d8d121e737 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:57.086825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795733b-c77c-420d-9d15-13d8d121e737 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:57.091694 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb4d4d-7a15-4435-a26b-b358bcbc829c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:57.092431 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb4d4d-7a15-4435-a26b-b358bcbc829c admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:57.092723 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb4d4d-7a15-4435-a26b-b358bcbc829c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:57.093012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb4d4d-7a15-4435-a26b-b358bcbc829c admin admin] PATH_INFO: `/v3/projects/24b58344b7da4672b31f1c0891afeac7/users/a7bd270828bc424b91bebaf1c7a48e62/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:57.094998 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb4d4d-7a15-4435-a26b-b358bcbc829c admin admin] RBAC: Authorizing `identity:create_grant(project_id=24b58344b7da4672b31f1c0891afeac7, user_id=a7bd270828bc424b91bebaf1c7a48e62, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:57.097649 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb4d4d-7a15-4435-a26b-b358bcbc829c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:57.109851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1287/2599] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:17:57 2022] PUT /identity/v3/projects/24b58344b7da4672b31f1c0891afeac7/users/a7bd270828bc424b91bebaf1c7a48e62/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:17:57.122022 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e90f9-d7eb-45f8-8f62-e8c12c7f04ea None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:57.122416 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e90f9-d7eb-45f8-8f62-e8c12c7f04ea None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:57.122723 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e90f9-d7eb-45f8-8f62-e8c12c7f04ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:57.150142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-996e90f9-d7eb-45f8-8f62-e8c12c7f04ea None None] MFA Rules not processed for user `a7bd270828bc424b91bebaf1c7a48e62`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:17:57.151389 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-996e90f9-d7eb-45f8-8f62-e8c12c7f04ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:57.163071 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:57.224213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1313/2600] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:17:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:17:57.236119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb4d4d-7a15-4435-a26b-b358bcbc829c admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:17:57.237836 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b0cb4d4d-7a15-4435-a26b-b358bcbc829c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:17:57.243205 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:17:57.297792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb4d4d-7a15-4435-a26b-b358bcbc829c admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:17:57.301707 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b7752-c250-4113-bc83-98296ca82316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:17:57.302429 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b7752-c250-4113-bc83-98296ca82316 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:17:57.302765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b7752-c250-4113-bc83-98296ca82316 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:17:57.303046 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b7752-c250-4113-bc83-98296ca82316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:17:57.361235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b7752-c250-4113-bc83-98296ca82316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:17:57.362635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b7752-c250-4113-bc83-98296ca82316 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:17:57.371689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1288/2601] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:17:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:18:03.713240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e90f9-d7eb-45f8-8f62-e8c12c7f04ea None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:03.723946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e90f9-d7eb-45f8-8f62-e8c12c7f04ea None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:03.727550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621ac1d-8059-4dc3-a547-86202cb0c7ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:03.728158 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621ac1d-8059-4dc3-a547-86202cb0c7ba service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:03.728442 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621ac1d-8059-4dc3-a547-86202cb0c7ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:03.728719 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621ac1d-8059-4dc3-a547-86202cb0c7ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:03.739172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f621ac1d-8059-4dc3-a547-86202cb0c7ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:03.740190 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f621ac1d-8059-4dc3-a547-86202cb0c7ba service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:03.750647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1314/2602] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:18:07.367596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=47 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:10.629401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b7752-c250-4113-bc83-98296ca82316 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:10.691130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b7752-c250-4113-bc83-98296ca82316 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:10.695653 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271b9ae-8bc0-49ea-8f45-997dbefb1c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:10.696266 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271b9ae-8bc0-49ea-8f45-997dbefb1c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:10.696545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271b9ae-8bc0-49ea-8f45-997dbefb1c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:10.696819 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271b9ae-8bc0-49ea-8f45-997dbefb1c31 admin admin] PATH_INFO: `/v3/users/a474f3be7d514b39ada1a487ed76d957` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:10.697980 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271b9ae-8bc0-49ea-8f45-997dbefb1c31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a474f3be7d514b39ada1a487ed76d957)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:10.700450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271b9ae-8bc0-49ea-8f45-997dbefb1c31 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:10.867758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9271b9ae-8bc0-49ea-8f45-997dbefb1c31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a474f3be7d514b39ada1a487ed76d957'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:10.880326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9271b9ae-8bc0-49ea-8f45-997dbefb1c31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a474f3be7d514b39ada1a487ed76d957'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:10.887992 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9271b9ae-8bc0-49ea-8f45-997dbefb1c31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a474f3be7d514b39ada1a487ed76d957'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:10.956746 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9271b9ae-8bc0-49ea-8f45-997dbefb1c31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a474f3be7d514b39ada1a487ed76d957'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:10.957383 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9271b9ae-8bc0-49ea-8f45-997dbefb1c31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a474f3be7d514b39ada1a487ed76d957'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:10.963856 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1289/2603] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:18:10 2022] DELETE /identity/v3/users/a474f3be7d514b39ada1a487ed76d957 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:10.970069 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b981ff-7da3-4a16-9197-9cdc4bd3601f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:10.970069 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b981ff-7da3-4a16-9197-9cdc4bd3601f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:10.970069 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b981ff-7da3-4a16-9197-9cdc4bd3601f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:10.987628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-79b981ff-7da3-4a16-9197-9cdc4bd3601f None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:10.988839 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-79b981ff-7da3-4a16-9197-9cdc4bd3601f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:11.047326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1315/2604] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:11.057205 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271b9ae-8bc0-49ea-8f45-997dbefb1c31 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:11.059118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9271b9ae-8bc0-49ea-8f45-997dbefb1c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:11.115167 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271b9ae-8bc0-49ea-8f45-997dbefb1c31 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:11.118884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f87c70-3127-4c78-8bfa-072e1ef8f52f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:11.119423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f87c70-3127-4c78-8bfa-072e1ef8f52f service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:11.119712 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f87c70-3127-4c78-8bfa-072e1ef8f52f service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:11.119992 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f87c70-3127-4c78-8bfa-072e1ef8f52f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:11.129787 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f87c70-3127-4c78-8bfa-072e1ef8f52f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:11.131261 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f87c70-3127-4c78-8bfa-072e1ef8f52f service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:11.141337 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1290/2605] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:18:11.512337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b981ff-7da3-4a16-9197-9cdc4bd3601f None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:11.522744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-79b981ff-7da3-4a16-9197-9cdc4bd3601f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:11.525810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b981ff-7da3-4a16-9197-9cdc4bd3601f None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:11.530611 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f341090-c47d-4b9f-847e-453403ae5e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:11.531245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f341090-c47d-4b9f-847e-453403ae5e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:11.531535 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f341090-c47d-4b9f-847e-453403ae5e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:11.531811 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f341090-c47d-4b9f-847e-453403ae5e1a admin admin] PATH_INFO: `/v3/projects/0bfb0b4014b347af878e62d7be4d80e5` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:11.532975 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f341090-c47d-4b9f-847e-453403ae5e1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bfb0b4014b347af878e62d7be4d80e5)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:11.535029 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f341090-c47d-4b9f-847e-453403ae5e1a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:11.648138 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4f341090-c47d-4b9f-847e-453403ae5e1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bfb0b4014b347af878e62d7be4d80e5', 'request_id': 'req-4f341090-c47d-4b9f-847e-453403ae5e1a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:11.704967 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4f341090-c47d-4b9f-847e-453403ae5e1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bfb0b4014b347af878e62d7be4d80e5', 'request_id': 'req-4f341090-c47d-4b9f-847e-453403ae5e1a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:11.711729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4f341090-c47d-4b9f-847e-453403ae5e1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bfb0b4014b347af878e62d7be4d80e5', 'request_id': 'req-4f341090-c47d-4b9f-847e-453403ae5e1a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:11.722871 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4f341090-c47d-4b9f-847e-453403ae5e1a admin admin] The token cache is being invalidate because project 0bfb0b4014b347af878e62d7be4d80e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:18:11.723153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4f341090-c47d-4b9f-847e-453403ae5e1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:11.723804 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4f341090-c47d-4b9f-847e-453403ae5e1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:11.725259 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1316/2606] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:18:11 2022] DELETE /identity/v3/projects/0bfb0b4014b347af878e62d7be4d80e5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:11.729451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f87c70-3127-4c78-8bfa-072e1ef8f52f service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:11.739858 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f87c70-3127-4c78-8bfa-072e1ef8f52f service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:11.757560 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:11.796925 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d73c3-21de-4d7f-81c6-00b06da06a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:11.797661 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d73c3-21de-4d7f-81c6-00b06da06a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:11.798134 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d73c3-21de-4d7f-81c6-00b06da06a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:11.798425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d73c3-21de-4d7f-81c6-00b06da06a6c admin admin] PATH_INFO: `/v3/users/a7bd270828bc424b91bebaf1c7a48e62` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:11.799570 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d73c3-21de-4d7f-81c6-00b06da06a6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7bd270828bc424b91bebaf1c7a48e62)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:11.801569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d73c3-21de-4d7f-81c6-00b06da06a6c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:12.028410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-743d73c3-21de-4d7f-81c6-00b06da06a6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7bd270828bc424b91bebaf1c7a48e62'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:12.040413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-743d73c3-21de-4d7f-81c6-00b06da06a6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7bd270828bc424b91bebaf1c7a48e62'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:12.047505 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-743d73c3-21de-4d7f-81c6-00b06da06a6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7bd270828bc424b91bebaf1c7a48e62'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:12.127711 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-743d73c3-21de-4d7f-81c6-00b06da06a6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7bd270828bc424b91bebaf1c7a48e62'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:12.128383 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-743d73c3-21de-4d7f-81c6-00b06da06a6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7bd270828bc424b91bebaf1c7a48e62'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:12.134912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1291/2607] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:18:11 2022] DELETE /identity/v3/users/a7bd270828bc424b91bebaf1c7a48e62 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:12.541757 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f341090-c47d-4b9f-847e-453403ae5e1a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:12.543667 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4f341090-c47d-4b9f-847e-453403ae5e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:12.567357 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f341090-c47d-4b9f-847e-453403ae5e1a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:12.585148 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:12.626722 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d80fa3-6f57-4632-a1ea-ed8aed1a3440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:12.627365 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d80fa3-6f57-4632-a1ea-ed8aed1a3440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:12.627647 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d80fa3-6f57-4632-a1ea-ed8aed1a3440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:12.627917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d80fa3-6f57-4632-a1ea-ed8aed1a3440 admin admin] PATH_INFO: `/v3/projects/24b58344b7da4672b31f1c0891afeac7` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:12.629068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d80fa3-6f57-4632-a1ea-ed8aed1a3440 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24b58344b7da4672b31f1c0891afeac7)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:12.631065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d80fa3-6f57-4632-a1ea-ed8aed1a3440 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:12.723512 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-66d80fa3-6f57-4632-a1ea-ed8aed1a3440 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24b58344b7da4672b31f1c0891afeac7', 'request_id': 'req-66d80fa3-6f57-4632-a1ea-ed8aed1a3440'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:12.802126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-66d80fa3-6f57-4632-a1ea-ed8aed1a3440 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24b58344b7da4672b31f1c0891afeac7', 'request_id': 'req-66d80fa3-6f57-4632-a1ea-ed8aed1a3440'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:12.809471 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-66d80fa3-6f57-4632-a1ea-ed8aed1a3440 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24b58344b7da4672b31f1c0891afeac7', 'request_id': 'req-66d80fa3-6f57-4632-a1ea-ed8aed1a3440'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:12.820602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-66d80fa3-6f57-4632-a1ea-ed8aed1a3440 admin admin] The token cache is being invalidate because project 24b58344b7da4672b31f1c0891afeac7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:18:12.820895 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-66d80fa3-6f57-4632-a1ea-ed8aed1a3440 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:12.821545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-66d80fa3-6f57-4632-a1ea-ed8aed1a3440 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:12.823189 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1317/2608] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:18:12 2022] DELETE /identity/v3/projects/24b58344b7da4672b31f1c0891afeac7 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:12.831146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc258dc-7a66-442f-b13b-8df48a1d7242 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:12.831642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc258dc-7a66-442f-b13b-8df48a1d7242 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:12.831761 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc258dc-7a66-442f-b13b-8df48a1d7242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:12.851963 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-bcc258dc-7a66-442f-b13b-8df48a1d7242 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:12.853315 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-bcc258dc-7a66-442f-b13b-8df48a1d7242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:12.913427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1292/2609] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:18:12.922245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7acd0-a269-4cf0-8e7f-a2a4424babc8 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:12.922545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7acd0-a269-4cf0-8e7f-a2a4424babc8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:12.922807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7acd0-a269-4cf0-8e7f-a2a4424babc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:12.943008 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d1f7acd0-a269-4cf0-8e7f-a2a4424babc8 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:12.944287 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d1f7acd0-a269-4cf0-8e7f-a2a4424babc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:12.952745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1318/2610] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:12.962848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc258dc-7a66-442f-b13b-8df48a1d7242 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:12.973208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc258dc-7a66-442f-b13b-8df48a1d7242 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:12.977726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182012e2-3849-4ea5-a8fa-259e34d7fdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:12.978365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182012e2-3849-4ea5-a8fa-259e34d7fdb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:12.978645 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182012e2-3849-4ea5-a8fa-259e34d7fdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:12.978924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182012e2-3849-4ea5-a8fa-259e34d7fdb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:12.979566 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182012e2-3849-4ea5-a8fa-259e34d7fdb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:18:12.979895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182012e2-3849-4ea5-a8fa-259e34d7fdb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:12.981497 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182012e2-3849-4ea5-a8fa-259e34d7fdb3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:12.992843 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1293/2611] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:18:12.997340 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7acd0-a269-4cf0-8e7f-a2a4424babc8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:13.006646 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7acd0-a269-4cf0-8e7f-a2a4424babc8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:13.011926 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066200d9-8752-40d7-a7d9-43c1a6d9a82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:13.012594 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066200d9-8752-40d7-a7d9-43c1a6d9a82a admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:13.012874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066200d9-8752-40d7-a7d9-43c1a6d9a82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:13.013195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066200d9-8752-40d7-a7d9-43c1a6d9a82a admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:13.013836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066200d9-8752-40d7-a7d9-43c1a6d9a82a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:13.015874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066200d9-8752-40d7-a7d9-43c1a6d9a82a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:13.111905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1319/2612] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:18:12 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:13.115945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182012e2-3849-4ea5-a8fa-259e34d7fdb3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:13.126498 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182012e2-3849-4ea5-a8fa-259e34d7fdb3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:13.153623 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:13.183888 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47851c26-e5ba-4c95-8fb2-54ee4e6b73d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:13.184505 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47851c26-e5ba-4c95-8fb2-54ee4e6b73d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:13.185016 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47851c26-e5ba-4c95-8fb2-54ee4e6b73d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:13.185518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47851c26-e5ba-4c95-8fb2-54ee4e6b73d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:13.186254 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47851c26-e5ba-4c95-8fb2-54ee4e6b73d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:13.188638 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47851c26-e5ba-4c95-8fb2-54ee4e6b73d8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:13.305536 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1294/2613] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:18:13 2022] POST /identity/v3/users => generated 287 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:13.312992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066200d9-8752-40d7-a7d9-43c1a6d9a82a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:13.323209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066200d9-8752-40d7-a7d9-43c1a6d9a82a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:13.328507 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988eb08-6b1d-4816-83d5-edbbd2888c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:13.329090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988eb08-6b1d-4816-83d5-edbbd2888c83 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:13.329371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988eb08-6b1d-4816-83d5-edbbd2888c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:13.329644 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988eb08-6b1d-4816-83d5-edbbd2888c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:13.330320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988eb08-6b1d-4816-83d5-edbbd2888c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:13.331917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988eb08-6b1d-4816-83d5-edbbd2888c83 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:13.342068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1320/2614] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:13.346105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47851c26-e5ba-4c95-8fb2-54ee4e6b73d8 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:13.356080 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47851c26-e5ba-4c95-8fb2-54ee4e6b73d8 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:13.360716 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe95918-7ad8-4669-ac78-481f49924e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:13.361312 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe95918-7ad8-4669-ac78-481f49924e2a admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:13.361600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe95918-7ad8-4669-ac78-481f49924e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:13.361960 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe95918-7ad8-4669-ac78-481f49924e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:13.362583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe95918-7ad8-4669-ac78-481f49924e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:13.364179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe95918-7ad8-4669-ac78-481f49924e2a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:13.375314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1295/2615] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:13.379376 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988eb08-6b1d-4816-83d5-edbbd2888c83 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:13.389829 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988eb08-6b1d-4816-83d5-edbbd2888c83 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:13.394420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a51972-be62-4d3e-91c1-3dbdaca0c51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:13.395058 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a51972-be62-4d3e-91c1-3dbdaca0c51e admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:13.395343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a51972-be62-4d3e-91c1-3dbdaca0c51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:13.395621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a51972-be62-4d3e-91c1-3dbdaca0c51e admin admin] PATH_INFO: `/v3/projects/4851838e9ef1424d9324a1d9ceeda7f2/users/0dea5a091d244c3f81d57d0ccd8dd26c/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:13.410090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a51972-be62-4d3e-91c1-3dbdaca0c51e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4851838e9ef1424d9324a1d9ceeda7f2, user_id=0dea5a091d244c3f81d57d0ccd8dd26c, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:13.412725 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a51972-be62-4d3e-91c1-3dbdaca0c51e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:13.476054 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1321/2616] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:18:13 2022] PUT /identity/v3/projects/4851838e9ef1424d9324a1d9ceeda7f2/users/0dea5a091d244c3f81d57d0ccd8dd26c/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:13.489059 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579e06e-72ba-4e0d-8108-f10b5fa7ee1c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:13.489379 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579e06e-72ba-4e0d-8108-f10b5fa7ee1c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:13.489657 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579e06e-72ba-4e0d-8108-f10b5fa7ee1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:13.517821 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b579e06e-72ba-4e0d-8108-f10b5fa7ee1c None None] MFA Rules not processed for user `0dea5a091d244c3f81d57d0ccd8dd26c`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:13.527582 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b579e06e-72ba-4e0d-8108-f10b5fa7ee1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:13.581827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1296/2617] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:13.593321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a51972-be62-4d3e-91c1-3dbdaca0c51e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:13.595095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c0a51972-be62-4d3e-91c1-3dbdaca0c51e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:13.647494 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:13.652659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a51972-be62-4d3e-91c1-3dbdaca0c51e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:13.656383 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182fe6c-af7e-47d5-a784-a347e22a7490 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:13.659140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182fe6c-af7e-47d5-a784-a347e22a7490 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:13.659140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182fe6c-af7e-47d5-a784-a347e22a7490 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:13.659140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182fe6c-af7e-47d5-a784-a347e22a7490 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:13.665876 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182fe6c-af7e-47d5-a784-a347e22a7490 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:13.667286 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182fe6c-af7e-47d5-a784-a347e22a7490 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:13.676299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1322/2618] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:18:13.701496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579e06e-72ba-4e0d-8108-f10b5fa7ee1c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:13.767502 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579e06e-72ba-4e0d-8108-f10b5fa7ee1c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:13.772644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b5dc1-4657-449b-9b12-bbd5a4b8990e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:13.773281 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b5dc1-4657-449b-9b12-bbd5a4b8990e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:13.773570 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b5dc1-4657-449b-9b12-bbd5a4b8990e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:13.773852 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b5dc1-4657-449b-9b12-bbd5a4b8990e admin admin] PATH_INFO: `/v3/users/0dea5a091d244c3f81d57d0ccd8dd26c` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:13.775230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b5dc1-4657-449b-9b12-bbd5a4b8990e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dea5a091d244c3f81d57d0ccd8dd26c)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:13.777689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b5dc1-4657-449b-9b12-bbd5a4b8990e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:13.925324 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:13.931784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a06b5dc1-4657-449b-9b12-bbd5a4b8990e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dea5a091d244c3f81d57d0ccd8dd26c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:13.943666 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a06b5dc1-4657-449b-9b12-bbd5a4b8990e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dea5a091d244c3f81d57d0ccd8dd26c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:13.950684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a06b5dc1-4657-449b-9b12-bbd5a4b8990e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dea5a091d244c3f81d57d0ccd8dd26c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:14.031013 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a06b5dc1-4657-449b-9b12-bbd5a4b8990e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dea5a091d244c3f81d57d0ccd8dd26c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:14.031979 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a06b5dc1-4657-449b-9b12-bbd5a4b8990e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dea5a091d244c3f81d57d0ccd8dd26c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:14.038514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1297/2619] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:18:13 2022] DELETE /identity/v3/users/0dea5a091d244c3f81d57d0ccd8dd26c => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:14.042988 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ed760-ddac-47f0-a303-5062faab596e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:14.043281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ed760-ddac-47f0-a303-5062faab596e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:14.043559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ed760-ddac-47f0-a303-5062faab596e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:14.062963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0c8ed760-ddac-47f0-a303-5062faab596e None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:14.064174 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0c8ed760-ddac-47f0-a303-5062faab596e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:14.127565 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1323/2620] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:14.138224 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b5dc1-4657-449b-9b12-bbd5a4b8990e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:14.139726 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a06b5dc1-4657-449b-9b12-bbd5a4b8990e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:14.202138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b5dc1-4657-449b-9b12-bbd5a4b8990e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:14.205655 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015161b5-8283-4c66-8beb-430b211b2ed5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:14.206308 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015161b5-8283-4c66-8beb-430b211b2ed5 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:14.206608 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015161b5-8283-4c66-8beb-430b211b2ed5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:14.206869 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015161b5-8283-4c66-8beb-430b211b2ed5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:14.215906 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015161b5-8283-4c66-8beb-430b211b2ed5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:14.217339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015161b5-8283-4c66-8beb-430b211b2ed5 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:14.229639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1298/2621] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:18:14.683876 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ed760-ddac-47f0-a303-5062faab596e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:14.685366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0c8ed760-ddac-47f0-a303-5062faab596e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:14.698300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ed760-ddac-47f0-a303-5062faab596e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:14.704008 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6d11c-f5dd-4a94-81c4-2ae12fa731cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:14.704665 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6d11c-f5dd-4a94-81c4-2ae12fa731cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:14.704933 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6d11c-f5dd-4a94-81c4-2ae12fa731cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:14.705208 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6d11c-f5dd-4a94-81c4-2ae12fa731cc admin admin] PATH_INFO: `/v3/projects/4851838e9ef1424d9324a1d9ceeda7f2` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:14.707974 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6d11c-f5dd-4a94-81c4-2ae12fa731cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4851838e9ef1424d9324a1d9ceeda7f2)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:14.708535 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6d11c-f5dd-4a94-81c4-2ae12fa731cc admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:14.826298 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dee6d11c-f5dd-4a94-81c4-2ae12fa731cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4851838e9ef1424d9324a1d9ceeda7f2', 'request_id': 'req-dee6d11c-f5dd-4a94-81c4-2ae12fa731cc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:14.888296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dee6d11c-f5dd-4a94-81c4-2ae12fa731cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4851838e9ef1424d9324a1d9ceeda7f2', 'request_id': 'req-dee6d11c-f5dd-4a94-81c4-2ae12fa731cc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:14.895089 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dee6d11c-f5dd-4a94-81c4-2ae12fa731cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4851838e9ef1424d9324a1d9ceeda7f2', 'request_id': 'req-dee6d11c-f5dd-4a94-81c4-2ae12fa731cc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:14.905911 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dee6d11c-f5dd-4a94-81c4-2ae12fa731cc admin admin] The token cache is being invalidate because project 4851838e9ef1424d9324a1d9ceeda7f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:18:14.906209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dee6d11c-f5dd-4a94-81c4-2ae12fa731cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:14.906844 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dee6d11c-f5dd-4a94-81c4-2ae12fa731cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:14.908312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1324/2622] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:18:14 2022] DELETE /identity/v3/projects/4851838e9ef1424d9324a1d9ceeda7f2 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:14.913375 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75a607-ed3f-42dc-a120-8b3a9828b55e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:14.913705 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75a607-ed3f-42dc-a120-8b3a9828b55e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:14.914021 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75a607-ed3f-42dc-a120-8b3a9828b55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:14.934280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-fa75a607-ed3f-42dc-a120-8b3a9828b55e None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:14.940062 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fa75a607-ed3f-42dc-a120-8b3a9828b55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:14.993516 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1299/2623] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:15.001979 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ecb7d-bb37-41f8-b3bf-67455f4524d6 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:15.002269 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ecb7d-bb37-41f8-b3bf-67455f4524d6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:15.002545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ecb7d-bb37-41f8-b3bf-67455f4524d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:15.023473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9e8ecb7d-bb37-41f8-b3bf-67455f4524d6 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:15.024803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9e8ecb7d-bb37-41f8-b3bf-67455f4524d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:15.032960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1325/2624] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:15.037428 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75a607-ed3f-42dc-a120-8b3a9828b55e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:15.048054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75a607-ed3f-42dc-a120-8b3a9828b55e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:15.052861 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc223d01-315e-4f92-8eae-18437f37992c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:15.053476 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc223d01-315e-4f92-8eae-18437f37992c admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:15.053762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc223d01-315e-4f92-8eae-18437f37992c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:15.054073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc223d01-315e-4f92-8eae-18437f37992c admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:15.054731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc223d01-315e-4f92-8eae-18437f37992c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:18:15.055043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc223d01-315e-4f92-8eae-18437f37992c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:15.056724 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc223d01-315e-4f92-8eae-18437f37992c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:15.069514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1300/2625] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:18:15.074095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ecb7d-bb37-41f8-b3bf-67455f4524d6 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:15.083438 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ecb7d-bb37-41f8-b3bf-67455f4524d6 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:15.088482 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bc848-a160-460e-ba3a-ef22d3dda395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:15.089088 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bc848-a160-460e-ba3a-ef22d3dda395 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:15.089368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bc848-a160-460e-ba3a-ef22d3dda395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:15.089640 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bc848-a160-460e-ba3a-ef22d3dda395 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:15.090406 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567bc848-a160-460e-ba3a-ef22d3dda395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:15.092295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567bc848-a160-460e-ba3a-ef22d3dda395 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:15.181458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1326/2626] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:18:15 2022] POST /identity/v3/projects => generated 397 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:15.186603 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc223d01-315e-4f92-8eae-18437f37992c admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:15.196730 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc223d01-315e-4f92-8eae-18437f37992c admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:15.240983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:15.254774 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48276812-6d76-4df3-85e8-c96f69724c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:15.255349 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48276812-6d76-4df3-85e8-c96f69724c74 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:15.255617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48276812-6d76-4df3-85e8-c96f69724c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:15.255901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48276812-6d76-4df3-85e8-c96f69724c74 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:15.256547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48276812-6d76-4df3-85e8-c96f69724c74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:15.258534 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48276812-6d76-4df3-85e8-c96f69724c74 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:15.345782 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1301/2627] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:18:15 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:15.350416 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bc848-a160-460e-ba3a-ef22d3dda395 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:15.361739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bc848-a160-460e-ba3a-ef22d3dda395 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:15.366336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f52988-c7e7-496c-a5a4-cf5abd3a3dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:15.366924 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f52988-c7e7-496c-a5a4-cf5abd3a3dca admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:15.367202 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f52988-c7e7-496c-a5a4-cf5abd3a3dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:15.367476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f52988-c7e7-496c-a5a4-cf5abd3a3dca admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:15.368157 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f52988-c7e7-496c-a5a4-cf5abd3a3dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:15.369722 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f52988-c7e7-496c-a5a4-cf5abd3a3dca admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:15.380144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1327/2628] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:15.390631 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48276812-6d76-4df3-85e8-c96f69724c74 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:15.396789 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48276812-6d76-4df3-85e8-c96f69724c74 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:15.402887 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f74d3-4358-4d79-a3c5-bd72f62a8ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:15.403537 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f74d3-4358-4d79-a3c5-bd72f62a8ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:15.403861 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f74d3-4358-4d79-a3c5-bd72f62a8ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:15.404140 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f74d3-4358-4d79-a3c5-bd72f62a8ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:15.405011 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f74d3-4358-4d79-a3c5-bd72f62a8ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:15.406779 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f74d3-4358-4d79-a3c5-bd72f62a8ef4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:15.419157 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1302/2629] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:15.423679 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f52988-c7e7-496c-a5a4-cf5abd3a3dca admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:15.434112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f52988-c7e7-496c-a5a4-cf5abd3a3dca admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:15.439095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee9808-99dd-4882-97e4-6c074cecca38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:15.439755 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee9808-99dd-4882-97e4-6c074cecca38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:15.440041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee9808-99dd-4882-97e4-6c074cecca38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:15.440346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee9808-99dd-4882-97e4-6c074cecca38 admin admin] PATH_INFO: `/v3/projects/bdc8f406a2bf4f8388b0757af168acc6/users/d52e90e4e64446469525c64374296ba6/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:15.456125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee9808-99dd-4882-97e4-6c074cecca38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc8f406a2bf4f8388b0757af168acc6, user_id=d52e90e4e64446469525c64374296ba6, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:15.458781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee9808-99dd-4882-97e4-6c074cecca38 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:15.534784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1328/2630] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:18:15 2022] PUT /identity/v3/projects/bdc8f406a2bf4f8388b0757af168acc6/users/d52e90e4e64446469525c64374296ba6/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:15.545895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f45268-9849-4e2f-b2ac-321fc2a34cc2 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:15.546282 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f45268-9849-4e2f-b2ac-321fc2a34cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:15.547316 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f45268-9849-4e2f-b2ac-321fc2a34cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:15.576582 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f3f45268-9849-4e2f-b2ac-321fc2a34cc2 None None] MFA Rules not processed for user `d52e90e4e64446469525c64374296ba6`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:15.578009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f3f45268-9849-4e2f-b2ac-321fc2a34cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:15.649200 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1303/2631] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:15 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:15.660946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee9808-99dd-4882-97e4-6c074cecca38 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:15.662682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ccee9808-99dd-4882-97e4-6c074cecca38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:15.674427 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:15.726942 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee9808-99dd-4882-97e4-6c074cecca38 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:15.731516 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339b351-31d0-4c12-8ffa-8539cb34b4bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:15.732161 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339b351-31d0-4c12-8ffa-8539cb34b4bb service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:15.732446 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339b351-31d0-4c12-8ffa-8539cb34b4bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:15.732720 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339b351-31d0-4c12-8ffa-8539cb34b4bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:15.741881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339b351-31d0-4c12-8ffa-8539cb34b4bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:15.743346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339b351-31d0-4c12-8ffa-8539cb34b4bb service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:15.752321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1329/2632] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:15 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:18:24.708751 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f45268-9849-4e2f-b2ac-321fc2a34cc2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:24.710256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f3f45268-9849-4e2f-b2ac-321fc2a34cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:24.780038 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f45268-9849-4e2f-b2ac-321fc2a34cc2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:24.785579 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83ac80-25fb-4ffd-b279-53fed2067401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:24.786553 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83ac80-25fb-4ffd-b279-53fed2067401 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:24.787031 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83ac80-25fb-4ffd-b279-53fed2067401 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:24.787493 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83ac80-25fb-4ffd-b279-53fed2067401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:24.790246 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0f83ac80-25fb-4ffd-b279-53fed2067401 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:24.863739 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83ac80-25fb-4ffd-b279-53fed2067401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:24.865166 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83ac80-25fb-4ffd-b279-53fed2067401 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:24.877609 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1304/2633] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:18:25.756344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=22 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:25.844170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339b351-31d0-4c12-8ffa-8539cb34b4bb service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:25.855581 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339b351-31d0-4c12-8ffa-8539cb34b4bb service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:25.860671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf91219-d3ae-47ae-b3eb-9abeb983777f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:25.861566 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf91219-d3ae-47ae-b3eb-9abeb983777f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:25.861879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf91219-d3ae-47ae-b3eb-9abeb983777f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:25.862168 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf91219-d3ae-47ae-b3eb-9abeb983777f admin admin] PATH_INFO: `/v3/users/d52e90e4e64446469525c64374296ba6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:25.863479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf91219-d3ae-47ae-b3eb-9abeb983777f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d52e90e4e64446469525c64374296ba6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:25.865487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf91219-d3ae-47ae-b3eb-9abeb983777f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:26.028557 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4bf91219-d3ae-47ae-b3eb-9abeb983777f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd52e90e4e64446469525c64374296ba6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:26.040734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4bf91219-d3ae-47ae-b3eb-9abeb983777f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd52e90e4e64446469525c64374296ba6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:26.049166 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4bf91219-d3ae-47ae-b3eb-9abeb983777f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd52e90e4e64446469525c64374296ba6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:26.100310 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4bf91219-d3ae-47ae-b3eb-9abeb983777f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd52e90e4e64446469525c64374296ba6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:26.102409 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4bf91219-d3ae-47ae-b3eb-9abeb983777f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd52e90e4e64446469525c64374296ba6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:26.111517 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1330/2634] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:18:25 2022] DELETE /identity/v3/users/d52e90e4e64446469525c64374296ba6 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:26.118049 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc6334-a8e2-4865-8c06-84fd547f8552 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:26.118049 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc6334-a8e2-4865-8c06-84fd547f8552 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:26.118049 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc6334-a8e2-4865-8c06-84fd547f8552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:26.126826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:26.139625 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-fffc6334-a8e2-4865-8c06-84fd547f8552 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:26.141202 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fffc6334-a8e2-4865-8c06-84fd547f8552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:26.218395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1305/2635] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:26.230442 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf91219-d3ae-47ae-b3eb-9abeb983777f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:26.232593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4bf91219-d3ae-47ae-b3eb-9abeb983777f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:26.305399 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf91219-d3ae-47ae-b3eb-9abeb983777f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:26.314834 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200e373-e6de-419a-a634-9c70d26afbd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:26.315597 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200e373-e6de-419a-a634-9c70d26afbd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:26.315969 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200e373-e6de-419a-a634-9c70d26afbd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:26.316313 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200e373-e6de-419a-a634-9c70d26afbd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:26.327050 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200e373-e6de-419a-a634-9c70d26afbd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:26.328715 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200e373-e6de-419a-a634-9c70d26afbd9 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:26.339884 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1331/2636] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:18:27.037821 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc6334-a8e2-4865-8c06-84fd547f8552 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:27.039321 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fffc6334-a8e2-4865-8c06-84fd547f8552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:27.051667 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc6334-a8e2-4865-8c06-84fd547f8552 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:27.056310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3506b9c-2517-4993-b307-971d4b7bc7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:27.056945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3506b9c-2517-4993-b307-971d4b7bc7e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:27.057236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3506b9c-2517-4993-b307-971d4b7bc7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:27.057516 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3506b9c-2517-4993-b307-971d4b7bc7e5 admin admin] PATH_INFO: `/v3/projects/bdc8f406a2bf4f8388b0757af168acc6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:27.058684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3506b9c-2517-4993-b307-971d4b7bc7e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc8f406a2bf4f8388b0757af168acc6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:27.060608 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3506b9c-2517-4993-b307-971d4b7bc7e5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:27.331473 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e3506b9c-2517-4993-b307-971d4b7bc7e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc8f406a2bf4f8388b0757af168acc6', 'request_id': 'req-e3506b9c-2517-4993-b307-971d4b7bc7e5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:27.419824 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e3506b9c-2517-4993-b307-971d4b7bc7e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc8f406a2bf4f8388b0757af168acc6', 'request_id': 'req-e3506b9c-2517-4993-b307-971d4b7bc7e5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:27.428115 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e3506b9c-2517-4993-b307-971d4b7bc7e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc8f406a2bf4f8388b0757af168acc6', 'request_id': 'req-e3506b9c-2517-4993-b307-971d4b7bc7e5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:27.441973 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e3506b9c-2517-4993-b307-971d4b7bc7e5 admin admin] The token cache is being invalidate because project bdc8f406a2bf4f8388b0757af168acc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:18:27.442494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e3506b9c-2517-4993-b307-971d4b7bc7e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:27.443333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e3506b9c-2517-4993-b307-971d4b7bc7e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:27.445389 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1306/2637] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:18:27 2022] DELETE /identity/v3/projects/bdc8f406a2bf4f8388b0757af168acc6 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:27.451105 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8b290-3fb0-4143-b2e3-07b7ea136c68 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:27.451656 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8b290-3fb0-4143-b2e3-07b7ea136c68 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:27.452017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8b290-3fb0-4143-b2e3-07b7ea136c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:27.473346 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3fe8b290-3fb0-4143-b2e3-07b7ea136c68 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:27.474699 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3fe8b290-3fb0-4143-b2e3-07b7ea136c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:27.546985 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1332/2638] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:27.556649 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915834c-a812-4aae-9e39-3aa01869ecb5 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:27.561407 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915834c-a812-4aae-9e39-3aa01869ecb5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:27.561407 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915834c-a812-4aae-9e39-3aa01869ecb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:27.578197 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4915834c-a812-4aae-9e39-3aa01869ecb5 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:27.579436 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4915834c-a812-4aae-9e39-3aa01869ecb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:27.587673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1307/2639] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:27.593040 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8b290-3fb0-4143-b2e3-07b7ea136c68 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:27.607136 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8b290-3fb0-4143-b2e3-07b7ea136c68 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:27.619611 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134895c-7b45-457b-9ae4-f1b900058ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:27.619611 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134895c-7b45-457b-9ae4-f1b900058ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:27.619611 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134895c-7b45-457b-9ae4-f1b900058ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:27.619611 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134895c-7b45-457b-9ae4-f1b900058ac4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:27.620304 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134895c-7b45-457b-9ae4-f1b900058ac4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:18:27.620304 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134895c-7b45-457b-9ae4-f1b900058ac4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:27.620304 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134895c-7b45-457b-9ae4-f1b900058ac4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:27.632618 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1333/2640] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 12:18:27.637759 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915834c-a812-4aae-9e39-3aa01869ecb5 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:27.654868 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915834c-a812-4aae-9e39-3aa01869ecb5 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:27.661597 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4d1c2-c11f-4f39-8665-aa07e461db83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:27.662262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4d1c2-c11f-4f39-8665-aa07e461db83 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:27.662544 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4d1c2-c11f-4f39-8665-aa07e461db83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:27.662820 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4d1c2-c11f-4f39-8665-aa07e461db83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:27.663492 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4d1c2-c11f-4f39-8665-aa07e461db83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:27.665450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4d1c2-c11f-4f39-8665-aa07e461db83 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:27.753426 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1308/2641] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:18:27 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:27.757775 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134895c-7b45-457b-9ae4-f1b900058ac4 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:27.763661 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:27.773033 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134895c-7b45-457b-9ae4-f1b900058ac4 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:27.840785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938afe4-cacb-4837-968f-2ea200fab48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:27.843684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938afe4-cacb-4837-968f-2ea200fab48e admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:27.843684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938afe4-cacb-4837-968f-2ea200fab48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:27.843684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938afe4-cacb-4837-968f-2ea200fab48e admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:27.843684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938afe4-cacb-4837-968f-2ea200fab48e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:27.845130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938afe4-cacb-4837-968f-2ea200fab48e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:27.946608 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1334/2642] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:18:27 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:27.951563 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4d1c2-c11f-4f39-8665-aa07e461db83 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:27.963686 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4d1c2-c11f-4f39-8665-aa07e461db83 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:27.968355 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e1f7a-a5c1-4054-9981-eb1c0548ac24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:27.969081 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e1f7a-a5c1-4054-9981-eb1c0548ac24 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:27.969368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e1f7a-a5c1-4054-9981-eb1c0548ac24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:27.969643 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e1f7a-a5c1-4054-9981-eb1c0548ac24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:27.970326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e1f7a-a5c1-4054-9981-eb1c0548ac24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:27.974364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e1f7a-a5c1-4054-9981-eb1c0548ac24 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:27.982691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1309/2643] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:18:27.988751 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938afe4-cacb-4837-968f-2ea200fab48e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:28.000153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938afe4-cacb-4837-968f-2ea200fab48e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:28.005142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad22c0-d8e1-4f44-8181-e78feed5b53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:28.005754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad22c0-d8e1-4f44-8181-e78feed5b53a admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:28.006151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad22c0-d8e1-4f44-8181-e78feed5b53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:28.006434 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad22c0-d8e1-4f44-8181-e78feed5b53a admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:28.007086 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad22c0-d8e1-4f44-8181-e78feed5b53a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:28.008680 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad22c0-d8e1-4f44-8181-e78feed5b53a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:28.020638 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1335/2644] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:18:28.025376 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e1f7a-a5c1-4054-9981-eb1c0548ac24 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:28.040848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e1f7a-a5c1-4054-9981-eb1c0548ac24 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:28.050036 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317c8c1-ddc5-46b3-ba95-32beba4fcbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:28.051203 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317c8c1-ddc5-46b3-ba95-32beba4fcbb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:28.051715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317c8c1-ddc5-46b3-ba95-32beba4fcbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:28.053144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317c8c1-ddc5-46b3-ba95-32beba4fcbb4 admin admin] PATH_INFO: `/v3/projects/9204365716924bbebdc0d90ecc13d01e/users/3b11628831fe4f50a7f9997c1f213a71/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:28.080263 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317c8c1-ddc5-46b3-ba95-32beba4fcbb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9204365716924bbebdc0d90ecc13d01e, user_id=3b11628831fe4f50a7f9997c1f213a71, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:28.085768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317c8c1-ddc5-46b3-ba95-32beba4fcbb4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:28.177601 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1310/2645] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:18:28 2022] PUT /identity/v3/projects/9204365716924bbebdc0d90ecc13d01e/users/3b11628831fe4f50a7f9997c1f213a71/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:18:28.183048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff585545-f7d0-4cdf-b143-ab0f5715d601 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:28.183372 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff585545-f7d0-4cdf-b143-ab0f5715d601 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:28.183729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff585545-f7d0-4cdf-b143-ab0f5715d601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:28.211042 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ff585545-f7d0-4cdf-b143-ab0f5715d601 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:28.212324 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ff585545-f7d0-4cdf-b143-ab0f5715d601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:28.296304 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1336/2646] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:28.307488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317c8c1-ddc5-46b3-ba95-32beba4fcbb4 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:28.309270 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3317c8c1-ddc5-46b3-ba95-32beba4fcbb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:28.342929 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:28.388094 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317c8c1-ddc5-46b3-ba95-32beba4fcbb4 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:28.392801 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255a1a9-6c72-4495-a751-6825a73cfacf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:28.393441 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255a1a9-6c72-4495-a751-6825a73cfacf service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:28.393729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255a1a9-6c72-4495-a751-6825a73cfacf service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:28.394041 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255a1a9-6c72-4495-a751-6825a73cfacf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:28.407128 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9255a1a9-6c72-4495-a751-6825a73cfacf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:28.408492 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9255a1a9-6c72-4495-a751-6825a73cfacf service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:28.421446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1311/2647] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:18:28.653287 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff585545-f7d0-4cdf-b143-ab0f5715d601 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:28.653287 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ff585545-f7d0-4cdf-b143-ab0f5715d601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:28.746749 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff585545-f7d0-4cdf-b143-ab0f5715d601 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:28.751961 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a1f98-ed80-4a8e-a07d-c121c0d7f3eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca10dd7ed714edb8b87017deecc41b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:28.752565 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a1f98-ed80-4a8e-a07d-c121c0d7f3eb service placement] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:28.752852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a1f98-ed80-4a8e-a07d-c121c0d7f3eb service placement] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:28.753133 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a1f98-ed80-4a8e-a07d-c121c0d7f3eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:28.755100 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-074a1f98-ed80-4a8e-a07d-c121c0d7f3eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:28.768240 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a1f98-ed80-4a8e-a07d-c121c0d7f3eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:28.770101 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a1f98-ed80-4a8e-a07d-c121c0d7f3eb service placement] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:28.783499 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1337/2648] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:18:28.833386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255a1a9-6c72-4495-a751-6825a73cfacf service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:28.835970 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9255a1a9-6c72-4495-a751-6825a73cfacf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:28.851295 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255a1a9-6c72-4495-a751-6825a73cfacf service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:28.856430 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d1ad6-c903-43e5-8ba9-0018cab56d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:28.857061 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d1ad6-c903-43e5-8ba9-0018cab56d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:28.857354 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d1ad6-c903-43e5-8ba9-0018cab56d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:28.857637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d1ad6-c903-43e5-8ba9-0018cab56d0b admin admin] PATH_INFO: `/v3/users/28e44067e07d4501a840108f4dcf053c` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:28.858894 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d1ad6-c903-43e5-8ba9-0018cab56d0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=28e44067e07d4501a840108f4dcf053c)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:28.860825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d1ad6-c903-43e5-8ba9-0018cab56d0b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:29.098235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-857d1ad6-c903-43e5-8ba9-0018cab56d0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28e44067e07d4501a840108f4dcf053c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:29.112056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-857d1ad6-c903-43e5-8ba9-0018cab56d0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28e44067e07d4501a840108f4dcf053c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:29.120764 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-857d1ad6-c903-43e5-8ba9-0018cab56d0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28e44067e07d4501a840108f4dcf053c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:29.196574 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-857d1ad6-c903-43e5-8ba9-0018cab56d0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28e44067e07d4501a840108f4dcf053c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:29.197233 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-857d1ad6-c903-43e5-8ba9-0018cab56d0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28e44067e07d4501a840108f4dcf053c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:29.204104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1312/2649] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:18:28 2022] DELETE /identity/v3/users/28e44067e07d4501a840108f4dcf053c => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:29.449877 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a1f98-ed80-4a8e-a07d-c121c0d7f3eb service placement] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:29.460981 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a1f98-ed80-4a8e-a07d-c121c0d7f3eb service placement] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:29.470332 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:29.532617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea7759-6a32-4285-9e31-86db4391050d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:29.533428 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea7759-6a32-4285-9e31-86db4391050d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:29.534485 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea7759-6a32-4285-9e31-86db4391050d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:29.534485 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea7759-6a32-4285-9e31-86db4391050d admin admin] PATH_INFO: `/v3/projects/d410170ed3ed4ce2b64d367cbf77790a` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:29.535855 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea7759-6a32-4285-9e31-86db4391050d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d410170ed3ed4ce2b64d367cbf77790a)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:29.538700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea7759-6a32-4285-9e31-86db4391050d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:29.649847 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-58ea7759-6a32-4285-9e31-86db4391050d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd410170ed3ed4ce2b64d367cbf77790a', 'request_id': 'req-58ea7759-6a32-4285-9e31-86db4391050d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:29.746192 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-58ea7759-6a32-4285-9e31-86db4391050d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd410170ed3ed4ce2b64d367cbf77790a', 'request_id': 'req-58ea7759-6a32-4285-9e31-86db4391050d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:29.756247 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-58ea7759-6a32-4285-9e31-86db4391050d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd410170ed3ed4ce2b64d367cbf77790a', 'request_id': 'req-58ea7759-6a32-4285-9e31-86db4391050d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:29.769095 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-58ea7759-6a32-4285-9e31-86db4391050d admin admin] The token cache is being invalidate because project d410170ed3ed4ce2b64d367cbf77790a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:18:29.769389 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-58ea7759-6a32-4285-9e31-86db4391050d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:29.770479 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-58ea7759-6a32-4285-9e31-86db4391050d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:29.774452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1338/2650] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:18:29 2022] DELETE /identity/v3/projects/d410170ed3ed4ce2b64d367cbf77790a => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:29.776825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564c3ec-c6f3-426c-a185-360397ccff15 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:29.777319 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564c3ec-c6f3-426c-a185-360397ccff15 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:29.777994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564c3ec-c6f3-426c-a185-360397ccff15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:29.801928 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c564c3ec-c6f3-426c-a185-360397ccff15 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:29.803395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c564c3ec-c6f3-426c-a185-360397ccff15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:29.887809 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1313/2651] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:18:29.901197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e313d0-bf44-4efa-8173-1a53340ae443 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:29.901197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e313d0-bf44-4efa-8173-1a53340ae443 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:29.901197 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e313d0-bf44-4efa-8173-1a53340ae443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:29.923661 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-44e313d0-bf44-4efa-8173-1a53340ae443 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:29.924895 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-44e313d0-bf44-4efa-8173-1a53340ae443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:29.933985 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1339/2652] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:29.939139 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564c3ec-c6f3-426c-a185-360397ccff15 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:29.950081 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564c3ec-c6f3-426c-a185-360397ccff15 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:29.955110 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fda9b8-0491-49d2-ba28-268c38de96a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:29.955719 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fda9b8-0491-49d2-ba28-268c38de96a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:29.956313 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fda9b8-0491-49d2-ba28-268c38de96a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:29.956313 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fda9b8-0491-49d2-ba28-268c38de96a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:29.957379 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fda9b8-0491-49d2-ba28-268c38de96a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:18:29.957379 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fda9b8-0491-49d2-ba28-268c38de96a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:29.958987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fda9b8-0491-49d2-ba28-268c38de96a9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:29.972764 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1314/2653] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:18:29.976844 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e313d0-bf44-4efa-8173-1a53340ae443 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:29.987330 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e313d0-bf44-4efa-8173-1a53340ae443 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:29.992710 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ff98e6-e2c8-4baa-8449-c611feab43f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:29.993546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff98e6-e2c8-4baa-8449-c611feab43f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:29.993848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff98e6-e2c8-4baa-8449-c611feab43f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:29.997848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff98e6-e2c8-4baa-8449-c611feab43f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:29.997848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ff98e6-e2c8-4baa-8449-c611feab43f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:29.997848 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ff98e6-e2c8-4baa-8449-c611feab43f6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:30.082488 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1340/2654] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:18:29 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:30.083656 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fda9b8-0491-49d2-ba28-268c38de96a9 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:30.109826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fda9b8-0491-49d2-ba28-268c38de96a9 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:30.148919 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:30.190131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b583ec3-c051-446a-9608-c5e5068ec318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:30.190911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b583ec3-c051-446a-9608-c5e5068ec318 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:30.191407 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b583ec3-c051-446a-9608-c5e5068ec318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:30.191792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b583ec3-c051-446a-9608-c5e5068ec318 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:30.192555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b583ec3-c051-446a-9608-c5e5068ec318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:30.194740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b583ec3-c051-446a-9608-c5e5068ec318 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:30.273176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1315/2655] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:18:30 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:18:30.278056 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ff98e6-e2c8-4baa-8449-c611feab43f6 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:30.288555 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ff98e6-e2c8-4baa-8449-c611feab43f6 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:30.295067 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a253294-4c21-4638-80ae-03953359baf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:30.295676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a253294-4c21-4638-80ae-03953359baf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:30.295955 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a253294-4c21-4638-80ae-03953359baf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:30.296228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a253294-4c21-4638-80ae-03953359baf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:30.296871 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a253294-4c21-4638-80ae-03953359baf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:30.298661 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a253294-4c21-4638-80ae-03953359baf0 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:30.314082 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1341/2656] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:30.318732 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b583ec3-c051-446a-9608-c5e5068ec318 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:30.329972 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b583ec3-c051-446a-9608-c5e5068ec318 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:30.335853 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce939a8-41cf-4352-a00d-548915b3f822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:30.336651 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce939a8-41cf-4352-a00d-548915b3f822 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:30.337012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce939a8-41cf-4352-a00d-548915b3f822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:30.337365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce939a8-41cf-4352-a00d-548915b3f822 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:30.338272 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce939a8-41cf-4352-a00d-548915b3f822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:30.340637 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce939a8-41cf-4352-a00d-548915b3f822 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:30.354437 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1316/2657] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:30.364529 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a253294-4c21-4638-80ae-03953359baf0 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:30.372334 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a253294-4c21-4638-80ae-03953359baf0 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:30.378193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c797d5-f471-4a50-9d93-d9edcea40457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:30.379791 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c797d5-f471-4a50-9d93-d9edcea40457 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:30.380118 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c797d5-f471-4a50-9d93-d9edcea40457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:30.380454 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c797d5-f471-4a50-9d93-d9edcea40457 admin admin] PATH_INFO: `/v3/projects/834a2513fbef4bef831947a0345832f1/users/da4779b17adb478f8ba0e359aa08a5bb/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:30.396914 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c797d5-f471-4a50-9d93-d9edcea40457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834a2513fbef4bef831947a0345832f1, user_id=da4779b17adb478f8ba0e359aa08a5bb, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:30.401278 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c797d5-f471-4a50-9d93-d9edcea40457 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:30.472969 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1342/2658] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:18:30 2022] PUT /identity/v3/projects/834a2513fbef4bef831947a0345832f1/users/da4779b17adb478f8ba0e359aa08a5bb/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:18:30.484083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262566bc-ef18-49a1-a317-32e5047fdcb2 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:30.484476 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262566bc-ef18-49a1-a317-32e5047fdcb2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:30.484879 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262566bc-ef18-49a1-a317-32e5047fdcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:30.515991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-262566bc-ef18-49a1-a317-32e5047fdcb2 None None] MFA Rules not processed for user `da4779b17adb478f8ba0e359aa08a5bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:30.517380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-262566bc-ef18-49a1-a317-32e5047fdcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:30.601694 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1317/2659] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:30.613819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c797d5-f471-4a50-9d93-d9edcea40457 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:30.615760 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-16c797d5-f471-4a50-9d93-d9edcea40457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:30.668522 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:30.704325 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c797d5-f471-4a50-9d93-d9edcea40457 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:30.714126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276697df-154d-473c-9467-8c33fb8eb30d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:30.715016 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276697df-154d-473c-9467-8c33fb8eb30d service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:30.715418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276697df-154d-473c-9467-8c33fb8eb30d service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:30.715812 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276697df-154d-473c-9467-8c33fb8eb30d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:30.724697 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276697df-154d-473c-9467-8c33fb8eb30d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:30.726888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276697df-154d-473c-9467-8c33fb8eb30d service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:30.737425 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1343/2660] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:18:33.778519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadad37-5813-403b-995d-671fe726b01c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:33.779037 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadad37-5813-403b-995d-671fe726b01c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:33.779136 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadad37-5813-403b-995d-671fe726b01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:33.808694 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-dcadad37-5813-403b-995d-671fe726b01c None None] MFA Rules not processed for user `3b11628831fe4f50a7f9997c1f213a71`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:33.809843 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-dcadad37-5813-403b-995d-671fe726b01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:33.889104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1318/2661] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:33.893724 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276697df-154d-473c-9467-8c33fb8eb30d service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:33.896023 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-276697df-154d-473c-9467-8c33fb8eb30d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:33.971620 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276697df-154d-473c-9467-8c33fb8eb30d service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:33.976895 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff76388-dbdd-4f86-af92-47926d10daba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:33.977462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff76388-dbdd-4f86-af92-47926d10daba admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:33.977745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff76388-dbdd-4f86-af92-47926d10daba admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:33.978079 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff76388-dbdd-4f86-af92-47926d10daba admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:33.978743 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff76388-dbdd-4f86-af92-47926d10daba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:33.980770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff76388-dbdd-4f86-af92-47926d10daba admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:34.080994 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1344/2662] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:18:33 2022] POST /identity/v3/projects => generated 369 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:34.086916 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadad37-5813-403b-995d-671fe726b01c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:34.104510 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadad37-5813-403b-995d-671fe726b01c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:34.139975 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:34.202314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa37155-b2a5-4ec8-949a-f8dc744afe4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:34.203335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa37155-b2a5-4ec8-949a-f8dc744afe4e admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:34.204075 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa37155-b2a5-4ec8-949a-f8dc744afe4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:34.204697 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa37155-b2a5-4ec8-949a-f8dc744afe4e admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:34.205447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa37155-b2a5-4ec8-949a-f8dc744afe4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:34.208919 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa37155-b2a5-4ec8-949a-f8dc744afe4e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:34.306251 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1319/2663] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:18:34 2022] POST /identity/v3/users => generated 291 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:18:34.310766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff76388-dbdd-4f86-af92-47926d10daba admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:34.321391 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff76388-dbdd-4f86-af92-47926d10daba admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:34.326932 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf718c-894c-4e6f-90e3-03077b909d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:34.327565 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf718c-894c-4e6f-90e3-03077b909d22 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:34.327850 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf718c-894c-4e6f-90e3-03077b909d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:34.328124 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf718c-894c-4e6f-90e3-03077b909d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:34.328960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf718c-894c-4e6f-90e3-03077b909d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:34.330967 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf718c-894c-4e6f-90e3-03077b909d22 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:34.348708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa37155-b2a5-4ec8-949a-f8dc744afe4e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:34.349379 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1345/2664] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:34.364175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa37155-b2a5-4ec8-949a-f8dc744afe4e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:34.369336 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de608b6-ff4f-46f0-bee3-f8117fe8f7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:34.370274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de608b6-ff4f-46f0-bee3-f8117fe8f7ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:34.370674 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de608b6-ff4f-46f0-bee3-f8117fe8f7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:34.371050 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de608b6-ff4f-46f0-bee3-f8117fe8f7ac admin admin] PATH_INFO: `/v3/projects/c1f845252ff945d0bd5cab90f3405462/users/a75421bca5c64ec2a40968085874ed60/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:34.390486 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de608b6-ff4f-46f0-bee3-f8117fe8f7ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f845252ff945d0bd5cab90f3405462, user_id=a75421bca5c64ec2a40968085874ed60, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:34.393336 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de608b6-ff4f-46f0-bee3-f8117fe8f7ac admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:34.472096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1320/2665] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:18:34 2022] PUT /identity/v3/projects/c1f845252ff945d0bd5cab90f3405462/users/a75421bca5c64ec2a40968085874ed60/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:18:34.476403 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf718c-894c-4e6f-90e3-03077b909d22 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:34.487933 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf718c-894c-4e6f-90e3-03077b909d22 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:34.583662 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86620d66-5d0d-47a3-a6e6-458c44f24cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:34.584373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86620d66-5d0d-47a3-a6e6-458c44f24cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:34.584738 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86620d66-5d0d-47a3-a6e6-458c44f24cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:34.585083 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86620d66-5d0d-47a3-a6e6-458c44f24cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:34.585823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86620d66-5d0d-47a3-a6e6-458c44f24cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:34.587518 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86620d66-5d0d-47a3-a6e6-458c44f24cc7 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:34.601900 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1346/2666] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:34.603091 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de608b6-ff4f-46f0-bee3-f8117fe8f7ac admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:34.619000 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de608b6-ff4f-46f0-bee3-f8117fe8f7ac admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:34.623995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15597d5-9edb-4439-8278-6b9206510dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:34.624717 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15597d5-9edb-4439-8278-6b9206510dc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:34.625005 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15597d5-9edb-4439-8278-6b9206510dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:34.625280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15597d5-9edb-4439-8278-6b9206510dc9 admin admin] PATH_INFO: `/v3/projects/c1f845252ff945d0bd5cab90f3405462/users/a75421bca5c64ec2a40968085874ed60/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:34.627557 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15597d5-9edb-4439-8278-6b9206510dc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f845252ff945d0bd5cab90f3405462, user_id=a75421bca5c64ec2a40968085874ed60, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:34.631649 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15597d5-9edb-4439-8278-6b9206510dc9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:34.643512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1321/2667] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:18:34 2022] PUT /identity/v3/projects/c1f845252ff945d0bd5cab90f3405462/users/a75421bca5c64ec2a40968085874ed60/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:40.205765 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75878fd1-282c-4703-8ee2-86e369462dae None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:40.206332 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75878fd1-282c-4703-8ee2-86e369462dae None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:40.206433 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75878fd1-282c-4703-8ee2-86e369462dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:40.239227 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-75878fd1-282c-4703-8ee2-86e369462dae None None] MFA Rules not processed for user `a75421bca5c64ec2a40968085874ed60`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:40.240476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-75878fd1-282c-4703-8ee2-86e369462dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:40.275312 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:40.344082 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1347/2668] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:40.359744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15597d5-9edb-4439-8278-6b9206510dc9 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:40.368023 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75878fd1-282c-4703-8ee2-86e369462dae None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:40.370168 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-75878fd1-282c-4703-8ee2-86e369462dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:40.441088 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15597d5-9edb-4439-8278-6b9206510dc9 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:40.450040 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b8194-61a7-4fbb-8364-47c85e015bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:40.450040 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b8194-61a7-4fbb-8364-47c85e015bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:40.450040 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b8194-61a7-4fbb-8364-47c85e015bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:40.450040 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b8194-61a7-4fbb-8364-47c85e015bfd admin admin] PATH_INFO: `/v3/users/da4779b17adb478f8ba0e359aa08a5bb` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:40.450492 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b8194-61a7-4fbb-8364-47c85e015bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=da4779b17adb478f8ba0e359aa08a5bb)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:40.451222 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b8194-61a7-4fbb-8364-47c85e015bfd admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:40.460603 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75878fd1-282c-4703-8ee2-86e369462dae None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:40.470344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1edbac-a2de-48a1-8242-5bd885f821fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:40.471327 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1edbac-a2de-48a1-8242-5bd885f821fa service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:40.471619 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1edbac-a2de-48a1-8242-5bd885f821fa service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:40.471898 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1edbac-a2de-48a1-8242-5bd885f821fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:40.545547 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1edbac-a2de-48a1-8242-5bd885f821fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:40.547166 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1edbac-a2de-48a1-8242-5bd885f821fa service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:40.556975 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1348/2669] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:18:40.602762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-619b8194-61a7-4fbb-8364-47c85e015bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da4779b17adb478f8ba0e359aa08a5bb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:40.616995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-619b8194-61a7-4fbb-8364-47c85e015bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da4779b17adb478f8ba0e359aa08a5bb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:40.624700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-619b8194-61a7-4fbb-8364-47c85e015bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da4779b17adb478f8ba0e359aa08a5bb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:40.629002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:40.730107 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-619b8194-61a7-4fbb-8364-47c85e015bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da4779b17adb478f8ba0e359aa08a5bb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:40.730866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-619b8194-61a7-4fbb-8364-47c85e015bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da4779b17adb478f8ba0e359aa08a5bb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:40.737898 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1322/2670] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:18:40 2022] DELETE /identity/v3/users/da4779b17adb478f8ba0e359aa08a5bb => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:40.742608 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a1c29-21c8-4d99-a967-698dfc2e23a0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:40.742896 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a1c29-21c8-4d99-a967-698dfc2e23a0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:40.743175 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a1c29-21c8-4d99-a967-698dfc2e23a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:40.764203 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ef7a1c29-21c8-4d99-a967-698dfc2e23a0 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:40.765463 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ef7a1c29-21c8-4d99-a967-698dfc2e23a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:40.838073 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:40.851436 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1349/2671] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:40.861704 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b8194-61a7-4fbb-8364-47c85e015bfd admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:40.863877 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-619b8194-61a7-4fbb-8364-47c85e015bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:40.937029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b8194-61a7-4fbb-8364-47c85e015bfd admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:40.941294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a145e-9178-4e13-96b7-6cde2e2fc7b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:40.950215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a145e-9178-4e13-96b7-6cde2e2fc7b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:40.950215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a145e-9178-4e13-96b7-6cde2e2fc7b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:40.950215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a145e-9178-4e13-96b7-6cde2e2fc7b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:40.953388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a145e-9178-4e13-96b7-6cde2e2fc7b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:40.954908 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a145e-9178-4e13-96b7-6cde2e2fc7b6 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:40.966947 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1323/2672] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:18:41.454616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a1c29-21c8-4d99-a967-698dfc2e23a0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:41.458063 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ef7a1c29-21c8-4d99-a967-698dfc2e23a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:41.472342 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a1c29-21c8-4d99-a967-698dfc2e23a0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:41.477615 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d42a34-287f-4000-a149-8d2535b446f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:41.478568 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d42a34-287f-4000-a149-8d2535b446f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:41.479031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d42a34-287f-4000-a149-8d2535b446f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:41.479501 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d42a34-287f-4000-a149-8d2535b446f1 admin admin] PATH_INFO: `/v3/projects/834a2513fbef4bef831947a0345832f1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:41.481084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d42a34-287f-4000-a149-8d2535b446f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=834a2513fbef4bef831947a0345832f1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:41.483261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d42a34-287f-4000-a149-8d2535b446f1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:41.624743 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-70d42a34-287f-4000-a149-8d2535b446f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '834a2513fbef4bef831947a0345832f1', 'request_id': 'req-70d42a34-287f-4000-a149-8d2535b446f1'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:41.704570 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-70d42a34-287f-4000-a149-8d2535b446f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '834a2513fbef4bef831947a0345832f1', 'request_id': 'req-70d42a34-287f-4000-a149-8d2535b446f1'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:41.711729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-70d42a34-287f-4000-a149-8d2535b446f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '834a2513fbef4bef831947a0345832f1', 'request_id': 'req-70d42a34-287f-4000-a149-8d2535b446f1'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:41.724813 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-70d42a34-287f-4000-a149-8d2535b446f1 admin admin] The token cache is being invalidate because project 834a2513fbef4bef831947a0345832f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:18:41.724813 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-70d42a34-287f-4000-a149-8d2535b446f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:41.724813 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-70d42a34-287f-4000-a149-8d2535b446f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:41.727368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1350/2673] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:18:41 2022] DELETE /identity/v3/projects/834a2513fbef4bef831947a0345832f1 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:41.731577 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c33483-a087-4c77-8ad8-6fcf865b8315 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:41.732065 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c33483-a087-4c77-8ad8-6fcf865b8315 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:41.732170 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c33483-a087-4c77-8ad8-6fcf865b8315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:41.761226 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-63c33483-a087-4c77-8ad8-6fcf865b8315 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:41.762926 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-63c33483-a087-4c77-8ad8-6fcf865b8315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:41.825527 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1324/2674] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:41.834594 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cdd51-84dd-412b-bf5c-5f7a249bc339 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:41.842069 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cdd51-84dd-412b-bf5c-5f7a249bc339 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:41.842069 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cdd51-84dd-412b-bf5c-5f7a249bc339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:41.855550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a58cdd51-84dd-412b-bf5c-5f7a249bc339 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:41.856771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a58cdd51-84dd-412b-bf5c-5f7a249bc339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:41.865125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1351/2675] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:41.871372 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c33483-a087-4c77-8ad8-6fcf865b8315 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:41.882611 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c33483-a087-4c77-8ad8-6fcf865b8315 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:41.888040 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d39ac-f80d-4a63-beaf-b475ca5838df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:41.888648 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d39ac-f80d-4a63-beaf-b475ca5838df admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:41.888928 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d39ac-f80d-4a63-beaf-b475ca5838df admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:41.890743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d39ac-f80d-4a63-beaf-b475ca5838df admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:41.890743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d39ac-f80d-4a63-beaf-b475ca5838df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:18:41.890743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d39ac-f80d-4a63-beaf-b475ca5838df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:41.891803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d39ac-f80d-4a63-beaf-b475ca5838df admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:41.904764 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1325/2676] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:18:41.909285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cdd51-84dd-412b-bf5c-5f7a249bc339 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:41.919649 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cdd51-84dd-412b-bf5c-5f7a249bc339 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:41.924418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9288f-c70b-4b88-9b2c-e027d9d20f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:41.925019 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9288f-c70b-4b88-9b2c-e027d9d20f7c admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:41.925301 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9288f-c70b-4b88-9b2c-e027d9d20f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:41.925572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9288f-c70b-4b88-9b2c-e027d9d20f7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:41.926274 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9288f-c70b-4b88-9b2c-e027d9d20f7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:41.928187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9288f-c70b-4b88-9b2c-e027d9d20f7c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:42.009178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1352/2677] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:18:41 2022] POST /identity/v3/projects => generated 403 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:42.013600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d39ac-f80d-4a63-beaf-b475ca5838df admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:42.024281 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d39ac-f80d-4a63-beaf-b475ca5838df admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:42.084670 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9190f-c87c-425e-8484-090b2b5a30ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:42.085290 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9190f-c87c-425e-8484-090b2b5a30ed admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:42.085571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9190f-c87c-425e-8484-090b2b5a30ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:42.085952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9190f-c87c-425e-8484-090b2b5a30ed admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:42.086590 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9190f-c87c-425e-8484-090b2b5a30ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:42.088653 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9190f-c87c-425e-8484-090b2b5a30ed admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:42.225221 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1326/2678] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:18:42 2022] POST /identity/v3/users => generated 302 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:42.229770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9288f-c70b-4b88-9b2c-e027d9d20f7c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:42.240481 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9288f-c70b-4b88-9b2c-e027d9d20f7c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:42.246616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db1514-f7e3-4338-aa7e-4b3d3721c36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:42.246799 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db1514-f7e3-4338-aa7e-4b3d3721c36a admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:42.248162 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db1514-f7e3-4338-aa7e-4b3d3721c36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:42.248162 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db1514-f7e3-4338-aa7e-4b3d3721c36a admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:42.248344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db1514-f7e3-4338-aa7e-4b3d3721c36a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:42.249945 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db1514-f7e3-4338-aa7e-4b3d3721c36a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:42.260734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1353/2679] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:42.265054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9190f-c87c-425e-8484-090b2b5a30ed admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:42.279180 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9190f-c87c-425e-8484-090b2b5a30ed admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:42.286384 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b598e3-7259-48ff-8035-2b922a23913a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:42.291334 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b598e3-7259-48ff-8035-2b922a23913a admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:42.291334 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b598e3-7259-48ff-8035-2b922a23913a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:42.291334 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b598e3-7259-48ff-8035-2b922a23913a admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:42.291334 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b598e3-7259-48ff-8035-2b922a23913a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:42.291334 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b598e3-7259-48ff-8035-2b922a23913a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:42.304200 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1327/2680] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:42.308734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db1514-f7e3-4338-aa7e-4b3d3721c36a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:42.320211 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db1514-f7e3-4338-aa7e-4b3d3721c36a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:42.325368 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedc7d8-650a-4fe2-a4fc-34c3424e7a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:42.326155 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedc7d8-650a-4fe2-a4fc-34c3424e7a9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:42.326467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedc7d8-650a-4fe2-a4fc-34c3424e7a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:42.326727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedc7d8-650a-4fe2-a4fc-34c3424e7a9b admin admin] PATH_INFO: `/v3/projects/a221c681aeb54e158e935434e7ef6cf1/users/663cc1ad3f394209a41232826aa8f2bc/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:42.345242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedc7d8-650a-4fe2-a4fc-34c3424e7a9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a221c681aeb54e158e935434e7ef6cf1, user_id=663cc1ad3f394209a41232826aa8f2bc, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:42.348422 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedc7d8-650a-4fe2-a4fc-34c3424e7a9b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:42.425659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1354/2681] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:18:42 2022] PUT /identity/v3/projects/a221c681aeb54e158e935434e7ef6cf1/users/663cc1ad3f394209a41232826aa8f2bc/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:18:42.436525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9581f83-ef0b-46d9-8229-376b297e8099 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:42.436824 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9581f83-ef0b-46d9-8229-376b297e8099 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:42.437106 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9581f83-ef0b-46d9-8229-376b297e8099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:42.445641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:42.474274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f9581f83-ef0b-46d9-8229-376b297e8099 None None] MFA Rules not processed for user `663cc1ad3f394209a41232826aa8f2bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:42.475930 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f9581f83-ef0b-46d9-8229-376b297e8099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:42.556602 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1328/2682] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:42 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:42.561184 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedc7d8-650a-4fe2-a4fc-34c3424e7a9b admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:42.638355 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedc7d8-650a-4fe2-a4fc-34c3424e7a9b admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:42.643462 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cc291-43f4-4256-becc-2697e4b1b5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:42.644235 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cc291-43f4-4256-becc-2697e4b1b5aa admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:42.644589 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cc291-43f4-4256-becc-2697e4b1b5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:42.644937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cc291-43f4-4256-becc-2697e4b1b5aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:42.645661 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5cc291-43f4-4256-becc-2697e4b1b5aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:42.648039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5cc291-43f4-4256-becc-2697e4b1b5aa admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:42.746524 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:42.754047 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1355/2683] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:18:42 2022] POST /identity/v3/projects => generated 399 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:42.763787 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9581f83-ef0b-46d9-8229-376b297e8099 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:42.776962 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9581f83-ef0b-46d9-8229-376b297e8099 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:42.867298 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ed4cf-7396-4993-981f-5c5193b81280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:42.867934 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ed4cf-7396-4993-981f-5c5193b81280 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:42.868218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ed4cf-7396-4993-981f-5c5193b81280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:42.868793 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ed4cf-7396-4993-981f-5c5193b81280 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:42.869514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ed4cf-7396-4993-981f-5c5193b81280 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:42.871514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ed4cf-7396-4993-981f-5c5193b81280 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:42.965162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1329/2684] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:18:42 2022] POST /identity/v3/users => generated 306 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:18:42.968128 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cc291-43f4-4256-becc-2697e4b1b5aa admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:42.982710 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cc291-43f4-4256-becc-2697e4b1b5aa admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:42.990268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae30719-e155-4db4-be6d-f99f00b37f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:42.990944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae30719-e155-4db4-be6d-f99f00b37f0e admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:42.991229 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae30719-e155-4db4-be6d-f99f00b37f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:42.991504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae30719-e155-4db4-be6d-f99f00b37f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:42.992404 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae30719-e155-4db4-be6d-f99f00b37f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:42.994596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae30719-e155-4db4-be6d-f99f00b37f0e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:43.012171 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1356/2685] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:43.019494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ed4cf-7396-4993-981f-5c5193b81280 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:43.024298 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ed4cf-7396-4993-981f-5c5193b81280 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:43.029684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a0084-28f8-4584-afe7-9c0fc5160bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:43.030506 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a0084-28f8-4584-afe7-9c0fc5160bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:43.030922 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a0084-28f8-4584-afe7-9c0fc5160bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:43.031210 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a0084-28f8-4584-afe7-9c0fc5160bd4 admin admin] PATH_INFO: `/v3/projects/9ef0e051aa1f48cf9a969f8331970ac9/users/88c03b3e5a254de99550c6ef517ff791/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:43.048706 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a0084-28f8-4584-afe7-9c0fc5160bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ef0e051aa1f48cf9a969f8331970ac9, user_id=88c03b3e5a254de99550c6ef517ff791, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:43.051559 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a0084-28f8-4584-afe7-9c0fc5160bd4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:43.135829 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1330/2686] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:18:43 2022] PUT /identity/v3/projects/9ef0e051aa1f48cf9a969f8331970ac9/users/88c03b3e5a254de99550c6ef517ff791/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:43.139680 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae30719-e155-4db4-be6d-f99f00b37f0e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:43.149824 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae30719-e155-4db4-be6d-f99f00b37f0e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:43.209255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c33b47-a06a-412b-8e55-a65bd03515fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:43.210085 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c33b47-a06a-412b-8e55-a65bd03515fb admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:43.210374 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c33b47-a06a-412b-8e55-a65bd03515fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:43.210653 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c33b47-a06a-412b-8e55-a65bd03515fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:43.211301 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c33b47-a06a-412b-8e55-a65bd03515fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:43.212993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c33b47-a06a-412b-8e55-a65bd03515fb admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:43.224002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1357/2687] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:43.228157 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a0084-28f8-4584-afe7-9c0fc5160bd4 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:43.239584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a0084-28f8-4584-afe7-9c0fc5160bd4 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:43.244283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4bc776-b038-4ed3-acc0-cd6a62ca43bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:43.244962 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bc776-b038-4ed3-acc0-cd6a62ca43bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:43.245249 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bc776-b038-4ed3-acc0-cd6a62ca43bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:43.245592 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bc776-b038-4ed3-acc0-cd6a62ca43bd admin admin] PATH_INFO: `/v3/projects/9ef0e051aa1f48cf9a969f8331970ac9/users/88c03b3e5a254de99550c6ef517ff791/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:43.248229 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bc776-b038-4ed3-acc0-cd6a62ca43bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ef0e051aa1f48cf9a969f8331970ac9, user_id=88c03b3e5a254de99550c6ef517ff791, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:43.251126 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bc776-b038-4ed3-acc0-cd6a62ca43bd admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:43.273729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1331/2688] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:18:43 2022] PUT /identity/v3/projects/9ef0e051aa1f48cf9a969f8331970ac9/users/88c03b3e5a254de99550c6ef517ff791/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:43.275793 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a512aa-45da-48a5-8b96-72e7f1b0f8b7 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:43.276636 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a512aa-45da-48a5-8b96-72e7f1b0f8b7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:43.276927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a512aa-45da-48a5-8b96-72e7f1b0f8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:43.308087 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-99a512aa-45da-48a5-8b96-72e7f1b0f8b7 None None] MFA Rules not processed for user `88c03b3e5a254de99550c6ef517ff791`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:43.309420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-99a512aa-45da-48a5-8b96-72e7f1b0f8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:43.398759 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1358/2689] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:43.411345 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a512aa-45da-48a5-8b96-72e7f1b0f8b7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:43.411554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff0354-6762-470f-9b83-06670f0b11b0 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:43.411937 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff0354-6762-470f-9b83-06670f0b11b0 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:43.412277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff0354-6762-470f-9b83-06670f0b11b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:43.413950 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-99a512aa-45da-48a5-8b96-72e7f1b0f8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:43.447365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-89ff0354-6762-470f-9b83-06670f0b11b0 None None] MFA Rules not processed for user `3b11628831fe4f50a7f9997c1f213a71`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:43.448893 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-89ff0354-6762-470f-9b83-06670f0b11b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:43.491730 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a512aa-45da-48a5-8b96-72e7f1b0f8b7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:43.497786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4d409-9ee3-4a93-8881-d1e3b41b6747 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:43.498814 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4d409-9ee3-4a93-8881-d1e3b41b6747 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:43.499839 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4d409-9ee3-4a93-8881-d1e3b41b6747 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:43.500399 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4d409-9ee3-4a93-8881-d1e3b41b6747 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:43.511293 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4d409-9ee3-4a93-8881-d1e3b41b6747 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:43.513213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4d409-9ee3-4a93-8881-d1e3b41b6747 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:43.514758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:43.525288 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1359/2690] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:43 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:18:43.532316 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1332/2691] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:43.705249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4d409-9ee3-4a93-8881-d1e3b41b6747 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:43.723155 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:43.783268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4d409-9ee3-4a93-8881-d1e3b41b6747 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:43.787996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72ced9-ce84-424f-b77d-108e769d6c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:43.788635 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72ced9-ce84-424f-b77d-108e769d6c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:43.788927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72ced9-ce84-424f-b77d-108e769d6c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:43.789210 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72ced9-ce84-424f-b77d-108e769d6c02 admin admin] PATH_INFO: `/v3/users/663cc1ad3f394209a41232826aa8f2bc` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:43.790795 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72ced9-ce84-424f-b77d-108e769d6c02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=663cc1ad3f394209a41232826aa8f2bc)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:43.792387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72ced9-ce84-424f-b77d-108e769d6c02 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:44.019288 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7e72ced9-ce84-424f-b77d-108e769d6c02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '663cc1ad3f394209a41232826aa8f2bc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:44.035075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7e72ced9-ce84-424f-b77d-108e769d6c02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '663cc1ad3f394209a41232826aa8f2bc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:44.043177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7e72ced9-ce84-424f-b77d-108e769d6c02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '663cc1ad3f394209a41232826aa8f2bc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:44.144689 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7e72ced9-ce84-424f-b77d-108e769d6c02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '663cc1ad3f394209a41232826aa8f2bc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:44.145302 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7e72ced9-ce84-424f-b77d-108e769d6c02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '663cc1ad3f394209a41232826aa8f2bc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:44.156500 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1360/2692] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:18:43 2022] DELETE /identity/v3/users/663cc1ad3f394209a41232826aa8f2bc => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:44.158162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4bd57-8e57-410a-a026-15069b08abaa None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:44.158699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4bd57-8e57-410a-a026-15069b08abaa None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:44.158977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4bd57-8e57-410a-a026-15069b08abaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:44.180782 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-67d4bd57-8e57-410a-a026-15069b08abaa None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:44.182207 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-67d4bd57-8e57-410a-a026-15069b08abaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:44.257947 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1333/2693] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:18:44.267574 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72ced9-ce84-424f-b77d-108e769d6c02 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:44.269215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7e72ced9-ce84-424f-b77d-108e769d6c02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:44.348073 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72ced9-ce84-424f-b77d-108e769d6c02 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:44.352234 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a5f3b-2b9f-403c-86c3-e737e9401557 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:44.352836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a5f3b-2b9f-403c-86c3-e737e9401557 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:44.353126 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a5f3b-2b9f-403c-86c3-e737e9401557 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:44.353548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a5f3b-2b9f-403c-86c3-e737e9401557 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:44.364629 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a5f3b-2b9f-403c-86c3-e737e9401557 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:44.366354 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a5f3b-2b9f-403c-86c3-e737e9401557 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:44.377691 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1361/2694] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:18:44.861235 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4bd57-8e57-410a-a026-15069b08abaa None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:44.862840 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-67d4bd57-8e57-410a-a026-15069b08abaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:44.876142 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4bd57-8e57-410a-a026-15069b08abaa None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:44.881102 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f49d7b-8ecb-419e-b575-1905bc3b3754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:44.881944 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f49d7b-8ecb-419e-b575-1905bc3b3754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:44.882239 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f49d7b-8ecb-419e-b575-1905bc3b3754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:44.882529 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f49d7b-8ecb-419e-b575-1905bc3b3754 admin admin] PATH_INFO: `/v3/projects/a221c681aeb54e158e935434e7ef6cf1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:44.883679 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f49d7b-8ecb-419e-b575-1905bc3b3754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a221c681aeb54e158e935434e7ef6cf1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:44.885701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f49d7b-8ecb-419e-b575-1905bc3b3754 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:45.003366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-20f49d7b-8ecb-419e-b575-1905bc3b3754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a221c681aeb54e158e935434e7ef6cf1', 'request_id': 'req-20f49d7b-8ecb-419e-b575-1905bc3b3754'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:45.092997 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-20f49d7b-8ecb-419e-b575-1905bc3b3754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a221c681aeb54e158e935434e7ef6cf1', 'request_id': 'req-20f49d7b-8ecb-419e-b575-1905bc3b3754'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:45.099347 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-20f49d7b-8ecb-419e-b575-1905bc3b3754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a221c681aeb54e158e935434e7ef6cf1', 'request_id': 'req-20f49d7b-8ecb-419e-b575-1905bc3b3754'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:45.113255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-20f49d7b-8ecb-419e-b575-1905bc3b3754 admin admin] The token cache is being invalidate because project a221c681aeb54e158e935434e7ef6cf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:18:45.113583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-20f49d7b-8ecb-419e-b575-1905bc3b3754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:45.114244 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-20f49d7b-8ecb-419e-b575-1905bc3b3754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:45.116985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1334/2695] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:18:44 2022] DELETE /identity/v3/projects/a221c681aeb54e158e935434e7ef6cf1 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:45.121614 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a5f3b-2b9f-403c-86c3-e737e9401557 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:45.132760 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a5f3b-2b9f-403c-86c3-e737e9401557 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:45.173026 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:45.200771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098b421-a6d1-4fde-aa08-c682f589d582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:45.201420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098b421-a6d1-4fde-aa08-c682f589d582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:45.201735 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098b421-a6d1-4fde-aa08-c682f589d582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:45.202055 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098b421-a6d1-4fde-aa08-c682f589d582 admin admin] PATH_INFO: `/v3/users/88c03b3e5a254de99550c6ef517ff791` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:45.203188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098b421-a6d1-4fde-aa08-c682f589d582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88c03b3e5a254de99550c6ef517ff791)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:45.205100 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098b421-a6d1-4fde-aa08-c682f589d582 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:45.389396 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4098b421-a6d1-4fde-aa08-c682f589d582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88c03b3e5a254de99550c6ef517ff791'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:45.403039 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4098b421-a6d1-4fde-aa08-c682f589d582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88c03b3e5a254de99550c6ef517ff791'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:45.410782 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4098b421-a6d1-4fde-aa08-c682f589d582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88c03b3e5a254de99550c6ef517ff791'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:45.498958 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4098b421-a6d1-4fde-aa08-c682f589d582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88c03b3e5a254de99550c6ef517ff791'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:45.499775 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4098b421-a6d1-4fde-aa08-c682f589d582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88c03b3e5a254de99550c6ef517ff791'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:45.506964 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1362/2696] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:18:45 2022] DELETE /identity/v3/users/88c03b3e5a254de99550c6ef517ff791 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:18:45.987051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f49d7b-8ecb-419e-b575-1905bc3b3754 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:45.988563 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-20f49d7b-8ecb-419e-b575-1905bc3b3754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:46.002015 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f49d7b-8ecb-419e-b575-1905bc3b3754 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:46.069480 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f4a85-446f-4be6-92e7-3177d0f0ee64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:46.070211 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f4a85-446f-4be6-92e7-3177d0f0ee64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:46.070499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f4a85-446f-4be6-92e7-3177d0f0ee64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:46.070783 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f4a85-446f-4be6-92e7-3177d0f0ee64 admin admin] PATH_INFO: `/v3/projects/9ef0e051aa1f48cf9a969f8331970ac9` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:46.071950 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f4a85-446f-4be6-92e7-3177d0f0ee64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ef0e051aa1f48cf9a969f8331970ac9)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:46.073941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f4a85-446f-4be6-92e7-3177d0f0ee64 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:46.091562 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:46.214461 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-311f4a85-446f-4be6-92e7-3177d0f0ee64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ef0e051aa1f48cf9a969f8331970ac9', 'request_id': 'req-311f4a85-446f-4be6-92e7-3177d0f0ee64'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:46.310467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-311f4a85-446f-4be6-92e7-3177d0f0ee64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ef0e051aa1f48cf9a969f8331970ac9', 'request_id': 'req-311f4a85-446f-4be6-92e7-3177d0f0ee64'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:46.317493 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-311f4a85-446f-4be6-92e7-3177d0f0ee64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ef0e051aa1f48cf9a969f8331970ac9', 'request_id': 'req-311f4a85-446f-4be6-92e7-3177d0f0ee64'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:46.329672 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-311f4a85-446f-4be6-92e7-3177d0f0ee64 admin admin] The token cache is being invalidate because project 9ef0e051aa1f48cf9a969f8331970ac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:18:46.330013 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-311f4a85-446f-4be6-92e7-3177d0f0ee64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:46.330788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-311f4a85-446f-4be6-92e7-3177d0f0ee64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:46.332462 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1335/2697] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:18:45 2022] DELETE /identity/v3/projects/9ef0e051aa1f48cf9a969f8331970ac9 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:46.337725 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ada48-32d6-4433-9e4a-6d8da00f121d None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:46.338384 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ada48-32d6-4433-9e4a-6d8da00f121d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:46.338384 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ada48-32d6-4433-9e4a-6d8da00f121d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:46.361277 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-c87ada48-32d6-4433-9e4a-6d8da00f121d None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:46.362581 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c87ada48-32d6-4433-9e4a-6d8da00f121d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:46.433545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1363/2698] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:46.443099 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e84ae0-2b31-4655-9723-71518eb37bdc None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:46.443395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e84ae0-2b31-4655-9723-71518eb37bdc None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:46.443678 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e84ae0-2b31-4655-9723-71518eb37bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:46.466357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-36e84ae0-2b31-4655-9723-71518eb37bdc None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:46.467695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-36e84ae0-2b31-4655-9723-71518eb37bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:46.477084 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1336/2699] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:46.481537 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ada48-32d6-4433-9e4a-6d8da00f121d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:46.491545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ada48-32d6-4433-9e4a-6d8da00f121d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:46.496221 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c5108-a21d-43a3-a404-0e4dbd855043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:46.496818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c5108-a21d-43a3-a404-0e4dbd855043 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:46.497104 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c5108-a21d-43a3-a404-0e4dbd855043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:46.497383 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c5108-a21d-43a3-a404-0e4dbd855043 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:46.498164 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c5108-a21d-43a3-a404-0e4dbd855043 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:18:46.498669 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c5108-a21d-43a3-a404-0e4dbd855043 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:46.500150 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c5108-a21d-43a3-a404-0e4dbd855043 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:46.512570 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1364/2700] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:18:46.517115 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e84ae0-2b31-4655-9723-71518eb37bdc None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:46.528160 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e84ae0-2b31-4655-9723-71518eb37bdc None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:46.533717 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f79cf5-5afd-4b6e-9217-6e096a67afda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:46.534502 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f79cf5-5afd-4b6e-9217-6e096a67afda admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:46.534918 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f79cf5-5afd-4b6e-9217-6e096a67afda admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:46.535198 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f79cf5-5afd-4b6e-9217-6e096a67afda admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:46.536051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f79cf5-5afd-4b6e-9217-6e096a67afda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:46.538606 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f79cf5-5afd-4b6e-9217-6e096a67afda admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:46.635302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1337/2701] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:18:46 2022] POST /identity/v3/projects => generated 401 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:46.637061 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c5108-a21d-43a3-a404-0e4dbd855043 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:46.652471 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c5108-a21d-43a3-a404-0e4dbd855043 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:46.694651 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:46.711327 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311d254-e72d-4bd3-bcbb-8a517960b548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:46.711936 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311d254-e72d-4bd3-bcbb-8a517960b548 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:46.712216 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311d254-e72d-4bd3-bcbb-8a517960b548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:46.712490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311d254-e72d-4bd3-bcbb-8a517960b548 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:46.713520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311d254-e72d-4bd3-bcbb-8a517960b548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:46.715655 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311d254-e72d-4bd3-bcbb-8a517960b548 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:46.797657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1365/2702] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:18:46 2022] POST /identity/v3/users => generated 301 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:46.801081 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f79cf5-5afd-4b6e-9217-6e096a67afda admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:46.812190 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f79cf5-5afd-4b6e-9217-6e096a67afda admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:46.816823 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ba9e6-bac4-45a2-b1bd-b62b254766da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:46.817483 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ba9e6-bac4-45a2-b1bd-b62b254766da admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:46.817775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ba9e6-bac4-45a2-b1bd-b62b254766da admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:46.818081 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ba9e6-bac4-45a2-b1bd-b62b254766da admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:46.821689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ba9e6-bac4-45a2-b1bd-b62b254766da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:46.821689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ba9e6-bac4-45a2-b1bd-b62b254766da admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:46.834740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1338/2703] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:46.836048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311d254-e72d-4bd3-bcbb-8a517960b548 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:46.848685 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311d254-e72d-4bd3-bcbb-8a517960b548 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:46.854370 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ea132-2946-466d-963f-db8c13b9d845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:46.854991 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ea132-2946-466d-963f-db8c13b9d845 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:46.855342 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ea132-2946-466d-963f-db8c13b9d845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:46.855818 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ea132-2946-466d-963f-db8c13b9d845 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:46.856561 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ea132-2946-466d-963f-db8c13b9d845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:46.858361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ea132-2946-466d-963f-db8c13b9d845 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:46.873028 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1366/2704] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:46.877744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ba9e6-bac4-45a2-b1bd-b62b254766da admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:46.888909 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ba9e6-bac4-45a2-b1bd-b62b254766da admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:46.893632 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc9ddf-c08f-4fa0-a266-c8e56e0ec350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:46.894308 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc9ddf-c08f-4fa0-a266-c8e56e0ec350 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:46.894594 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc9ddf-c08f-4fa0-a266-c8e56e0ec350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:46.894873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc9ddf-c08f-4fa0-a266-c8e56e0ec350 admin admin] PATH_INFO: `/v3/projects/66e48e906d5f4ec5b7b0a7109b53f8ce/users/56002fa02ba64f10a7a1132ce9da255d/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:46.910083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc9ddf-c08f-4fa0-a266-c8e56e0ec350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66e48e906d5f4ec5b7b0a7109b53f8ce, user_id=56002fa02ba64f10a7a1132ce9da255d, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:46.912731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc9ddf-c08f-4fa0-a266-c8e56e0ec350 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:46.979597 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1339/2705] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:18:46 2022] PUT /identity/v3/projects/66e48e906d5f4ec5b7b0a7109b53f8ce/users/56002fa02ba64f10a7a1132ce9da255d/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:46.990035 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1769b7-41ba-4789-a847-3ac5b532d00b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:46.990327 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1769b7-41ba-4789-a847-3ac5b532d00b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:46.990638 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1769b7-41ba-4789-a847-3ac5b532d00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:47.022233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2e1769b7-41ba-4789-a847-3ac5b532d00b None None] MFA Rules not processed for user `56002fa02ba64f10a7a1132ce9da255d`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:47.023468 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2e1769b7-41ba-4789-a847-3ac5b532d00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:47.088556 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1367/2706] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:46 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:47.093384 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc9ddf-c08f-4fa0-a266-c8e56e0ec350 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:47.157283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc9ddf-c08f-4fa0-a266-c8e56e0ec350 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:47.161984 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bce55a-6f2d-466e-96e9-8be6c2ecec3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:47.162591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bce55a-6f2d-466e-96e9-8be6c2ecec3c admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:47.162877 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bce55a-6f2d-466e-96e9-8be6c2ecec3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:47.163163 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bce55a-6f2d-466e-96e9-8be6c2ecec3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:47.163841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bce55a-6f2d-466e-96e9-8be6c2ecec3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:47.165712 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bce55a-6f2d-466e-96e9-8be6c2ecec3c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:47.243967 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1340/2707] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:18:47 2022] POST /identity/v3/projects => generated 399 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:47.248680 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1769b7-41ba-4789-a847-3ac5b532d00b None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:47.259421 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1769b7-41ba-4789-a847-3ac5b532d00b None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:47.317708 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49608c6-38cf-4126-9ee6-891ca4c5d386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:47.318364 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49608c6-38cf-4126-9ee6-891ca4c5d386 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:47.318652 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49608c6-38cf-4126-9ee6-891ca4c5d386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:47.318935 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49608c6-38cf-4126-9ee6-891ca4c5d386 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:47.319579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49608c6-38cf-4126-9ee6-891ca4c5d386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:47.321501 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49608c6-38cf-4126-9ee6-891ca4c5d386 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:47.413830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1368/2708] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:18:47 2022] POST /identity/v3/users => generated 306 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:47.418745 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bce55a-6f2d-466e-96e9-8be6c2ecec3c admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:47.429460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bce55a-6f2d-466e-96e9-8be6c2ecec3c admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:47.435148 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20617d76-dd89-4d45-8a16-b217b0bb7013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:47.435758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20617d76-dd89-4d45-8a16-b217b0bb7013 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:47.436043 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20617d76-dd89-4d45-8a16-b217b0bb7013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:47.436343 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20617d76-dd89-4d45-8a16-b217b0bb7013 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:47.436995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20617d76-dd89-4d45-8a16-b217b0bb7013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:47.438634 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20617d76-dd89-4d45-8a16-b217b0bb7013 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:47.450903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1341/2709] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:47.455065 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49608c6-38cf-4126-9ee6-891ca4c5d386 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:47.465452 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49608c6-38cf-4126-9ee6-891ca4c5d386 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:47.470497 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d81c98-a99f-4da4-a78c-705d6826e3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:47.471188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d81c98-a99f-4da4-a78c-705d6826e3e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:47.471468 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d81c98-a99f-4da4-a78c-705d6826e3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:47.471740 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d81c98-a99f-4da4-a78c-705d6826e3e8 admin admin] PATH_INFO: `/v3/projects/4b029392e9f84ffeac7252992de5e080/users/d4a02b92c10449d1bd57dd9a1934826d/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:47.487357 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d81c98-a99f-4da4-a78c-705d6826e3e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b029392e9f84ffeac7252992de5e080, user_id=d4a02b92c10449d1bd57dd9a1934826d, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:47.490061 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d81c98-a99f-4da4-a78c-705d6826e3e8 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:47.591771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1369/2710] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:18:47 2022] PUT /identity/v3/projects/4b029392e9f84ffeac7252992de5e080/users/d4a02b92c10449d1bd57dd9a1934826d/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:47.599565 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20617d76-dd89-4d45-8a16-b217b0bb7013 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:47.611374 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20617d76-dd89-4d45-8a16-b217b0bb7013 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:47.622839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:47.672588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dd7df-2216-4e79-b611-bd7744b8e7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:47.673130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dd7df-2216-4e79-b611-bd7744b8e7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:47.673417 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dd7df-2216-4e79-b611-bd7744b8e7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:47.673697 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dd7df-2216-4e79-b611-bd7744b8e7c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:47.674391 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dd7df-2216-4e79-b611-bd7744b8e7c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:47.676007 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dd7df-2216-4e79-b611-bd7744b8e7c1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:47.687123 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1342/2711] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:47.691146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d81c98-a99f-4da4-a78c-705d6826e3e8 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:47.702492 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d81c98-a99f-4da4-a78c-705d6826e3e8 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:47.707213 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2890e-bedb-4ca7-a973-29ff5fc500f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:47.708038 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2890e-bedb-4ca7-a973-29ff5fc500f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:47.708317 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2890e-bedb-4ca7-a973-29ff5fc500f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:47.708591 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2890e-bedb-4ca7-a973-29ff5fc500f7 admin admin] PATH_INFO: `/v3/projects/4b029392e9f84ffeac7252992de5e080/users/d4a02b92c10449d1bd57dd9a1934826d/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:47.710522 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2890e-bedb-4ca7-a973-29ff5fc500f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b029392e9f84ffeac7252992de5e080, user_id=d4a02b92c10449d1bd57dd9a1934826d, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:47.713194 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2890e-bedb-4ca7-a973-29ff5fc500f7 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:47.725912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1370/2712] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:18:47 2022] PUT /identity/v3/projects/4b029392e9f84ffeac7252992de5e080/users/d4a02b92c10449d1bd57dd9a1934826d/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:18:47.737071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5610466-5ec4-4875-a5c5-a125b1658725 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:47.737421 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5610466-5ec4-4875-a5c5-a125b1658725 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:47.737624 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5610466-5ec4-4875-a5c5-a125b1658725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:47.765066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d5610466-5ec4-4875-a5c5-a125b1658725 None None] MFA Rules not processed for user `d4a02b92c10449d1bd57dd9a1934826d`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:47.766414 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d5610466-5ec4-4875-a5c5-a125b1658725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:47.838171 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1343/2713] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:47 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:47.851273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2890e-bedb-4ca7-a973-29ff5fc500f7 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:47.853309 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4de2890e-bedb-4ca7-a973-29ff5fc500f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:47.859594 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:47.914339 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2890e-bedb-4ca7-a973-29ff5fc500f7 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:47.918612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d4237-6795-4a6e-9408-15800826b9ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:47.919243 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d4237-6795-4a6e-9408-15800826b9ab service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:47.919528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d4237-6795-4a6e-9408-15800826b9ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:47.919805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d4237-6795-4a6e-9408-15800826b9ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:47.930282 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d4237-6795-4a6e-9408-15800826b9ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:47.931740 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d4237-6795-4a6e-9408-15800826b9ab service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:47.941624 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1371/2714] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:47 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:18:48.156108 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5610466-5ec4-4875-a5c5-a125b1658725 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:48.166207 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5610466-5ec4-4875-a5c5-a125b1658725 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:48.169992 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaafa64-a605-483a-83fe-074868b84d8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:48.170589 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaafa64-a605-483a-83fe-074868b84d8c service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:48.170869 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaafa64-a605-483a-83fe-074868b84d8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:48.171145 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaafa64-a605-483a-83fe-074868b84d8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:48.233337 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaafa64-a605-483a-83fe-074868b84d8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:48.234767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaafa64-a605-483a-83fe-074868b84d8c service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:48.243906 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1344/2715] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:48 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:18:54.418087 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d4237-6795-4a6e-9408-15800826b9ab service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:54.482352 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d4237-6795-4a6e-9408-15800826b9ab service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:54.487049 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d84d2-4155-475f-aed2-473bba470466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:54.487684 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d84d2-4155-475f-aed2-473bba470466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:54.487970 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d84d2-4155-475f-aed2-473bba470466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:54.488250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d84d2-4155-475f-aed2-473bba470466 admin admin] PATH_INFO: `/v3/users/56002fa02ba64f10a7a1132ce9da255d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:54.489398 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d84d2-4155-475f-aed2-473bba470466 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56002fa02ba64f10a7a1132ce9da255d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:54.491344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d84d2-4155-475f-aed2-473bba470466 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:54.662968 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ed3d84d2-4155-475f-aed2-473bba470466 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56002fa02ba64f10a7a1132ce9da255d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:54.675414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ed3d84d2-4155-475f-aed2-473bba470466 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56002fa02ba64f10a7a1132ce9da255d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:54.682261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ed3d84d2-4155-475f-aed2-473bba470466 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56002fa02ba64f10a7a1132ce9da255d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:54.750420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ed3d84d2-4155-475f-aed2-473bba470466 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56002fa02ba64f10a7a1132ce9da255d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:54.751042 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ed3d84d2-4155-475f-aed2-473bba470466 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56002fa02ba64f10a7a1132ce9da255d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:54.757467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1372/2716] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:18:54 2022] DELETE /identity/v3/users/56002fa02ba64f10a7a1132ce9da255d => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:54.762179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8e918-8b2e-453e-ad49-9281a1f677dc None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:54.762690 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8e918-8b2e-453e-ad49-9281a1f677dc None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:54.762792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8e918-8b2e-453e-ad49-9281a1f677dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:54.784861 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a1c8e918-8b2e-453e-ad49-9281a1f677dc None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:54.786122 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a1c8e918-8b2e-453e-ad49-9281a1f677dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:54.826044 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:54.858700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1345/2717] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:54.868252 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d84d2-4155-475f-aed2-473bba470466 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:54.869873 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ed3d84d2-4155-475f-aed2-473bba470466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:54.898413 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:54.929924 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d84d2-4155-475f-aed2-473bba470466 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:54.934255 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44516de7-1ce2-4ea3-8e6d-caf44c19a4a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:54.934873 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44516de7-1ce2-4ea3-8e6d-caf44c19a4a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:54.935160 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44516de7-1ce2-4ea3-8e6d-caf44c19a4a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:54.935453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44516de7-1ce2-4ea3-8e6d-caf44c19a4a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:54.945387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44516de7-1ce2-4ea3-8e6d-caf44c19a4a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:54.946909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44516de7-1ce2-4ea3-8e6d-caf44c19a4a0 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:54.957340 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1373/2718] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:18:55.415257 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8e918-8b2e-453e-ad49-9281a1f677dc None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:55.416686 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a1c8e918-8b2e-453e-ad49-9281a1f677dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:55.429982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8e918-8b2e-453e-ad49-9281a1f677dc None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:55.434199 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca45da8-69ee-4f10-ae47-8fb5e6501ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:55.434834 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca45da8-69ee-4f10-ae47-8fb5e6501ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:55.435182 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca45da8-69ee-4f10-ae47-8fb5e6501ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:55.435379 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca45da8-69ee-4f10-ae47-8fb5e6501ebe admin admin] PATH_INFO: `/v3/projects/66e48e906d5f4ec5b7b0a7109b53f8ce` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:55.436555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca45da8-69ee-4f10-ae47-8fb5e6501ebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=66e48e906d5f4ec5b7b0a7109b53f8ce)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:55.438534 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca45da8-69ee-4f10-ae47-8fb5e6501ebe admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:55.603531 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-eca45da8-69ee-4f10-ae47-8fb5e6501ebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66e48e906d5f4ec5b7b0a7109b53f8ce', 'request_id': 'req-eca45da8-69ee-4f10-ae47-8fb5e6501ebe'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:55.704701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-eca45da8-69ee-4f10-ae47-8fb5e6501ebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66e48e906d5f4ec5b7b0a7109b53f8ce', 'request_id': 'req-eca45da8-69ee-4f10-ae47-8fb5e6501ebe'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:55.711145 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-eca45da8-69ee-4f10-ae47-8fb5e6501ebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66e48e906d5f4ec5b7b0a7109b53f8ce', 'request_id': 'req-eca45da8-69ee-4f10-ae47-8fb5e6501ebe'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:55.723237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-eca45da8-69ee-4f10-ae47-8fb5e6501ebe admin admin] The token cache is being invalidate because project 66e48e906d5f4ec5b7b0a7109b53f8ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:18:55.723525 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-eca45da8-69ee-4f10-ae47-8fb5e6501ebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:55.724217 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-eca45da8-69ee-4f10-ae47-8fb5e6501ebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:55.725689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1346/2719] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:18:55 2022] DELETE /identity/v3/projects/66e48e906d5f4ec5b7b0a7109b53f8ce => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:55.729726 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44516de7-1ce2-4ea3-8e6d-caf44c19a4a0 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:55.740079 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44516de7-1ce2-4ea3-8e6d-caf44c19a4a0 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:55.797188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb549d6-8a21-4c85-951c-7fa7d3fac882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:55.797802 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb549d6-8a21-4c85-951c-7fa7d3fac882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:55.798112 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb549d6-8a21-4c85-951c-7fa7d3fac882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:55.798385 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb549d6-8a21-4c85-951c-7fa7d3fac882 admin admin] PATH_INFO: `/v3/users/d4a02b92c10449d1bd57dd9a1934826d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:55.799493 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb549d6-8a21-4c85-951c-7fa7d3fac882 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4a02b92c10449d1bd57dd9a1934826d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:55.801438 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb549d6-8a21-4c85-951c-7fa7d3fac882 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:56.083136 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0bb549d6-8a21-4c85-951c-7fa7d3fac882 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4a02b92c10449d1bd57dd9a1934826d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:56.096154 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0bb549d6-8a21-4c85-951c-7fa7d3fac882 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4a02b92c10449d1bd57dd9a1934826d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:56.103585 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0bb549d6-8a21-4c85-951c-7fa7d3fac882 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4a02b92c10449d1bd57dd9a1934826d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:56.224991 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0bb549d6-8a21-4c85-951c-7fa7d3fac882 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4a02b92c10449d1bd57dd9a1934826d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:56.225671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0bb549d6-8a21-4c85-951c-7fa7d3fac882 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4a02b92c10449d1bd57dd9a1934826d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:56.232352 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1374/2720] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:18:55 2022] DELETE /identity/v3/users/d4a02b92c10449d1bd57dd9a1934826d => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:56.694404 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca45da8-69ee-4f10-ae47-8fb5e6501ebe admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:56.695881 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-eca45da8-69ee-4f10-ae47-8fb5e6501ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:56.709241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca45da8-69ee-4f10-ae47-8fb5e6501ebe admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:56.772153 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d3c7a-a3d1-4239-992a-4c6c9cd1cdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:56.772793 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d3c7a-a3d1-4239-992a-4c6c9cd1cdb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:56.773077 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d3c7a-a3d1-4239-992a-4c6c9cd1cdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:56.773356 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d3c7a-a3d1-4239-992a-4c6c9cd1cdb3 admin admin] PATH_INFO: `/v3/projects/4b029392e9f84ffeac7252992de5e080` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:56.774508 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d3c7a-a3d1-4239-992a-4c6c9cd1cdb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b029392e9f84ffeac7252992de5e080)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:56.776414 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d3c7a-a3d1-4239-992a-4c6c9cd1cdb3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:56.883083 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-132d3c7a-a3d1-4239-992a-4c6c9cd1cdb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b029392e9f84ffeac7252992de5e080', 'request_id': 'req-132d3c7a-a3d1-4239-992a-4c6c9cd1cdb3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:56.967504 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-132d3c7a-a3d1-4239-992a-4c6c9cd1cdb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b029392e9f84ffeac7252992de5e080', 'request_id': 'req-132d3c7a-a3d1-4239-992a-4c6c9cd1cdb3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:56.972162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:56.974654 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-132d3c7a-a3d1-4239-992a-4c6c9cd1cdb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b029392e9f84ffeac7252992de5e080', 'request_id': 'req-132d3c7a-a3d1-4239-992a-4c6c9cd1cdb3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:56.986552 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-132d3c7a-a3d1-4239-992a-4c6c9cd1cdb3 admin admin] The token cache is being invalidate because project 4b029392e9f84ffeac7252992de5e080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:18:56.986845 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-132d3c7a-a3d1-4239-992a-4c6c9cd1cdb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:56.987626 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-132d3c7a-a3d1-4239-992a-4c6c9cd1cdb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:18:56.989335 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1347/2721] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:18:56 2022] DELETE /identity/v3/projects/4b029392e9f84ffeac7252992de5e080 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:56.994350 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cae88-b4c8-4e7b-9443-724d05b13194 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:56.994823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cae88-b4c8-4e7b-9443-724d05b13194 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:56.994936 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cae88-b4c8-4e7b-9443-724d05b13194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:57.014546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-077cae88-b4c8-4e7b-9443-724d05b13194 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:57.015764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-077cae88-b4c8-4e7b-9443-724d05b13194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:57.059110 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:57.076484 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1375/2722] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:57.084348 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db2e0f-0e7a-4671-b349-4570dcf2d2c8 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:57.084642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db2e0f-0e7a-4671-b349-4570dcf2d2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:57.084927 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db2e0f-0e7a-4671-b349-4570dcf2d2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:57.106220 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-86db2e0f-0e7a-4671-b349-4570dcf2d2c8 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:57.107449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-86db2e0f-0e7a-4671-b349-4570dcf2d2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:57.115564 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1348/2723] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:57.119876 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cae88-b4c8-4e7b-9443-724d05b13194 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:57.129600 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cae88-b4c8-4e7b-9443-724d05b13194 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:57.134254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de7af1-7543-44a9-bf5c-644161788e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:57.134857 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de7af1-7543-44a9-bf5c-644161788e77 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:57.135165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de7af1-7543-44a9-bf5c-644161788e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:57.135466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de7af1-7543-44a9-bf5c-644161788e77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:57.136106 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de7af1-7543-44a9-bf5c-644161788e77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:18:57.136412 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de7af1-7543-44a9-bf5c-644161788e77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:57.138057 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de7af1-7543-44a9-bf5c-644161788e77 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:57.150017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1376/2724] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:18:57.154791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db2e0f-0e7a-4671-b349-4570dcf2d2c8 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:57.164054 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db2e0f-0e7a-4671-b349-4570dcf2d2c8 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:57.169670 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0793a-ed19-48d5-b1fa-995db39a5d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:57.170334 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0793a-ed19-48d5-b1fa-995db39a5d90 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:57.170571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0793a-ed19-48d5-b1fa-995db39a5d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:57.170901 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0793a-ed19-48d5-b1fa-995db39a5d90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:57.171575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0793a-ed19-48d5-b1fa-995db39a5d90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:57.173519 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0793a-ed19-48d5-b1fa-995db39a5d90 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:57.312141 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1349/2725] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:18:57 2022] POST /identity/v3/projects => generated 379 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:57.316253 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de7af1-7543-44a9-bf5c-644161788e77 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:57.328080 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de7af1-7543-44a9-bf5c-644161788e77 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:57.382716 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338c830-3b12-40dd-b9f5-c39368a25968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:57.383318 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338c830-3b12-40dd-b9f5-c39368a25968 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:57.383602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338c830-3b12-40dd-b9f5-c39368a25968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:57.383881 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338c830-3b12-40dd-b9f5-c39368a25968 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:57.384546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338c830-3b12-40dd-b9f5-c39368a25968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:57.386481 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338c830-3b12-40dd-b9f5-c39368a25968 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:57.482072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1377/2726] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:18:57 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:57.486381 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0793a-ed19-48d5-b1fa-995db39a5d90 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:57.496854 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0793a-ed19-48d5-b1fa-995db39a5d90 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:57.501587 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b9d45-b8ba-4445-9b2d-6a3d9f332ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:57.502220 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b9d45-b8ba-4445-9b2d-6a3d9f332ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:57.502504 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b9d45-b8ba-4445-9b2d-6a3d9f332ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:57.502781 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b9d45-b8ba-4445-9b2d-6a3d9f332ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:57.503476 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b9d45-b8ba-4445-9b2d-6a3d9f332ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:57.505062 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b9d45-b8ba-4445-9b2d-6a3d9f332ec2 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:57.515592 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1350/2727] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:57.523637 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338c830-3b12-40dd-b9f5-c39368a25968 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:57.536003 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338c830-3b12-40dd-b9f5-c39368a25968 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:57.540195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3454060-7d3f-4b84-91d5-5a132ff0ee23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:57.540783 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3454060-7d3f-4b84-91d5-5a132ff0ee23 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:57.541059 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3454060-7d3f-4b84-91d5-5a132ff0ee23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:57.541360 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3454060-7d3f-4b84-91d5-5a132ff0ee23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:57.542035 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3454060-7d3f-4b84-91d5-5a132ff0ee23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:57.543634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3454060-7d3f-4b84-91d5-5a132ff0ee23 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:57.554184 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1378/2728] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:57.558380 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b9d45-b8ba-4445-9b2d-6a3d9f332ec2 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:57.568550 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b9d45-b8ba-4445-9b2d-6a3d9f332ec2 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:57.573192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779474f3-e761-44d2-b590-cdec55215b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:57.573835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779474f3-e761-44d2-b590-cdec55215b02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:57.574146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779474f3-e761-44d2-b590-cdec55215b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:57.574977 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779474f3-e761-44d2-b590-cdec55215b02 admin admin] PATH_INFO: `/v3/projects/9398e0d8a00a4a8ca5f10d8eb3521395/users/6100e823d1c4464397c5e31a95e73e0a/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:57.589428 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779474f3-e761-44d2-b590-cdec55215b02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9398e0d8a00a4a8ca5f10d8eb3521395, user_id=6100e823d1c4464397c5e31a95e73e0a, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:57.592680 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779474f3-e761-44d2-b590-cdec55215b02 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:57.705527 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1351/2729] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:18:57 2022] PUT /identity/v3/projects/9398e0d8a00a4a8ca5f10d8eb3521395/users/6100e823d1c4464397c5e31a95e73e0a/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:57.715917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b54042-bfd9-4693-8706-0a280dc3092c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:57.716188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b54042-bfd9-4693-8706-0a280dc3092c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:57.716468 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b54042-bfd9-4693-8706-0a280dc3092c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:57.744631 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-90b54042-bfd9-4693-8706-0a280dc3092c None None] MFA Rules not processed for user `6100e823d1c4464397c5e31a95e73e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:57.745882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-90b54042-bfd9-4693-8706-0a280dc3092c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:57.815264 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1379/2730] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:57.821630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779474f3-e761-44d2-b590-cdec55215b02 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:57.891760 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779474f3-e761-44d2-b590-cdec55215b02 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:57.896447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb70a5-397d-438e-abed-6912ae463d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:57.897314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb70a5-397d-438e-abed-6912ae463d32 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:57.897627 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb70a5-397d-438e-abed-6912ae463d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:57.897920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb70a5-397d-438e-abed-6912ae463d32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:57.898556 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb70a5-397d-438e-abed-6912ae463d32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:57.900442 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb70a5-397d-438e-abed-6912ae463d32 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:58.193983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1352/2731] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:18:57 2022] POST /identity/v3/projects => generated 377 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:58.196675 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b54042-bfd9-4693-8706-0a280dc3092c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:58.207388 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b54042-bfd9-4693-8706-0a280dc3092c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:58.250351 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:58.267151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f70a6-0f9f-4176-ba17-08ef12b48fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:58.267764 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f70a6-0f9f-4176-ba17-08ef12b48fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:58.268046 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f70a6-0f9f-4176-ba17-08ef12b48fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:58.268317 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f70a6-0f9f-4176-ba17-08ef12b48fd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:58.268952 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f70a6-0f9f-4176-ba17-08ef12b48fd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:58.270922 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f70a6-0f9f-4176-ba17-08ef12b48fd2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:58.384468 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1380/2732] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:18:58 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:58.388630 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb70a5-397d-438e-abed-6912ae463d32 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:58.399539 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb70a5-397d-438e-abed-6912ae463d32 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:58.404904 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02098b0c-5fcd-4df2-a159-51b5c6c40faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:58.405523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02098b0c-5fcd-4df2-a159-51b5c6c40faf admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:58.405805 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02098b0c-5fcd-4df2-a159-51b5c6c40faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:58.406100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02098b0c-5fcd-4df2-a159-51b5c6c40faf admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:58.406742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02098b0c-5fcd-4df2-a159-51b5c6c40faf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:58.408328 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02098b0c-5fcd-4df2-a159-51b5c6c40faf admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:58.419581 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1353/2733] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:58.423589 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f70a6-0f9f-4176-ba17-08ef12b48fd2 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:58.433612 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f70a6-0f9f-4176-ba17-08ef12b48fd2 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:58.438296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e552b-5fa4-40a0-b152-33717ad1b3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:58.438891 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e552b-5fa4-40a0-b152-33717ad1b3eb admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:58.439172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e552b-5fa4-40a0-b152-33717ad1b3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:58.439446 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e552b-5fa4-40a0-b152-33717ad1b3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:58.440083 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e552b-5fa4-40a0-b152-33717ad1b3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:58.441668 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e552b-5fa4-40a0-b152-33717ad1b3eb admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:58.451552 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1381/2734] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:58.455561 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02098b0c-5fcd-4df2-a159-51b5c6c40faf admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:58.466306 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02098b0c-5fcd-4df2-a159-51b5c6c40faf admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:58.470889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9ae11-d9b7-488f-8a5f-bf708b992f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:58.471535 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9ae11-d9b7-488f-8a5f-bf708b992f4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:58.471822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9ae11-d9b7-488f-8a5f-bf708b992f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:58.472103 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9ae11-d9b7-488f-8a5f-bf708b992f4f admin admin] PATH_INFO: `/v3/projects/2e3fffbd8a6b437fb86ede09f52aa8c3/users/8ab26fd9a29c4869bc4a1f99d639a52e/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:58.487208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9ae11-d9b7-488f-8a5f-bf708b992f4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e3fffbd8a6b437fb86ede09f52aa8c3, user_id=8ab26fd9a29c4869bc4a1f99d639a52e, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:58.489843 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9ae11-d9b7-488f-8a5f-bf708b992f4f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:58.590989 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1354/2735] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:18:58 2022] PUT /identity/v3/projects/2e3fffbd8a6b437fb86ede09f52aa8c3/users/8ab26fd9a29c4869bc4a1f99d639a52e/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:58.602111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a357ed-2496-4bb6-99e0-50d9346b0100 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:58.606345 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a357ed-2496-4bb6-99e0-50d9346b0100 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:58.606345 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a357ed-2496-4bb6-99e0-50d9346b0100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:58.631146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-54a357ed-2496-4bb6-99e0-50d9346b0100 None None] MFA Rules not processed for user `8ab26fd9a29c4869bc4a1f99d639a52e`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:58.632366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-54a357ed-2496-4bb6-99e0-50d9346b0100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:58.696397 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1382/2736] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:58.700968 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9ae11-d9b7-488f-8a5f-bf708b992f4f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:58.742442 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:58.764721 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9ae11-d9b7-488f-8a5f-bf708b992f4f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:58.769514 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcf005-29c5-4798-b4c2-4d4fc4295896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:58.770143 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcf005-29c5-4798-b4c2-4d4fc4295896 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:58.770426 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcf005-29c5-4798-b4c2-4d4fc4295896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:58.770701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcf005-29c5-4798-b4c2-4d4fc4295896 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:58.771332 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffcf005-29c5-4798-b4c2-4d4fc4295896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:58.773742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffcf005-29c5-4798-b4c2-4d4fc4295896 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:58.895365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1355/2737] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:18:58 2022] POST /identity/v3/projects => generated 377 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:58.900596 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a357ed-2496-4bb6-99e0-50d9346b0100 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:58.912621 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a357ed-2496-4bb6-99e0-50d9346b0100 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:58.970288 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e081a-33fd-47c7-8130-e8d981a46b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:58.970912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e081a-33fd-47c7-8130-e8d981a46b01 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:58.971224 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e081a-33fd-47c7-8130-e8d981a46b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:58.971504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e081a-33fd-47c7-8130-e8d981a46b01 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:58.972159 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e081a-33fd-47c7-8130-e8d981a46b01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:58.974195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e081a-33fd-47c7-8130-e8d981a46b01 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:59.082544 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1383/2738] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:18:58 2022] POST /identity/v3/users => generated 295 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:18:59.086306 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcf005-29c5-4798-b4c2-4d4fc4295896 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:59.097570 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcf005-29c5-4798-b4c2-4d4fc4295896 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:59.102364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6534ee-fb93-4a89-9389-477daf23e97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:59.102949 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6534ee-fb93-4a89-9389-477daf23e97a admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:59.103237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6534ee-fb93-4a89-9389-477daf23e97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:59.103516 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6534ee-fb93-4a89-9389-477daf23e97a admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:59.104164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6534ee-fb93-4a89-9389-477daf23e97a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:59.105796 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6534ee-fb93-4a89-9389-477daf23e97a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:59.115723 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1356/2739] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:59.119938 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e081a-33fd-47c7-8130-e8d981a46b01 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:59.130508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e081a-33fd-47c7-8130-e8d981a46b01 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:59.135193 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319b0e0-d3a6-4edf-a611-2141f5274200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:59.135832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319b0e0-d3a6-4edf-a611-2141f5274200 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:59.136113 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319b0e0-d3a6-4edf-a611-2141f5274200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:59.136387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319b0e0-d3a6-4edf-a611-2141f5274200 admin admin] PATH_INFO: `/v3/projects/31b08f8253c049d7875660581f405d14/users/044fd434beab4f06acff4129ef561be8/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:59.152170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319b0e0-d3a6-4edf-a611-2141f5274200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b08f8253c049d7875660581f405d14, user_id=044fd434beab4f06acff4129ef561be8, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:59.154868 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319b0e0-d3a6-4edf-a611-2141f5274200 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:59.236457 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1384/2740] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:18:59 2022] PUT /identity/v3/projects/31b08f8253c049d7875660581f405d14/users/044fd434beab4f06acff4129ef561be8/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:59.240846 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6534ee-fb93-4a89-9389-477daf23e97a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:59.251848 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6534ee-fb93-4a89-9389-477daf23e97a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:59.310673 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1bda2-4f3e-4940-88fa-2caf9ced793e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:59.311281 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1bda2-4f3e-4940-88fa-2caf9ced793e admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:59.311564 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1bda2-4f3e-4940-88fa-2caf9ced793e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:59.311841 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1bda2-4f3e-4940-88fa-2caf9ced793e admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:59.312479 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1bda2-4f3e-4940-88fa-2caf9ced793e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:59.314089 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1bda2-4f3e-4940-88fa-2caf9ced793e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:59.325042 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1357/2741] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:18:59.329000 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319b0e0-d3a6-4edf-a611-2141f5274200 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:59.339783 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319b0e0-d3a6-4edf-a611-2141f5274200 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:59.344208 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ede28-ac82-4d57-83cb-3fb7aa1e32f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:59.344852 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ede28-ac82-4d57-83cb-3fb7aa1e32f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:59.345130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ede28-ac82-4d57-83cb-3fb7aa1e32f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:59.345404 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ede28-ac82-4d57-83cb-3fb7aa1e32f8 admin admin] PATH_INFO: `/v3/projects/31b08f8253c049d7875660581f405d14/users/044fd434beab4f06acff4129ef561be8/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:59.347322 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ede28-ac82-4d57-83cb-3fb7aa1e32f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b08f8253c049d7875660581f405d14, user_id=044fd434beab4f06acff4129ef561be8, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:59.349960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ede28-ac82-4d57-83cb-3fb7aa1e32f8 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:59.360426 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1385/2742] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:18:59 2022] PUT /identity/v3/projects/31b08f8253c049d7875660581f405d14/users/044fd434beab4f06acff4129ef561be8/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:18:59.370715 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbfd9c-8bd0-44cf-b168-11feb6e33db0 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:59.371539 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbfd9c-8bd0-44cf-b168-11feb6e33db0 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:59.371818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbfd9c-8bd0-44cf-b168-11feb6e33db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:59.401070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-47fbfd9c-8bd0-44cf-b168-11feb6e33db0 None None] MFA Rules not processed for user `044fd434beab4f06acff4129ef561be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:18:59.402356 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-47fbfd9c-8bd0-44cf-b168-11feb6e33db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:59.468962 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1358/2743] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:18:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:18:59.480561 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ede28-ac82-4d57-83cb-3fb7aa1e32f8 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:59.482249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8c4ede28-ac82-4d57-83cb-3fb7aa1e32f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:59.487706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:59.540090 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ede28-ac82-4d57-83cb-3fb7aa1e32f8 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:59.544271 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23ccfa3-22b4-4a83-b7df-9ff72ac7c296 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:59.544866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ccfa3-22b4-4a83-b7df-9ff72ac7c296 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:59.545146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ccfa3-22b4-4a83-b7df-9ff72ac7c296 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:59.545415 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ccfa3-22b4-4a83-b7df-9ff72ac7c296 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:59.554961 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ccfa3-22b4-4a83-b7df-9ff72ac7c296 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:59.556372 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ccfa3-22b4-4a83-b7df-9ff72ac7c296 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:59.566490 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1386/2744] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:18:59.676307 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbfd9c-8bd0-44cf-b168-11feb6e33db0 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:18:59.676307 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-47fbfd9c-8bd0-44cf-b168-11feb6e33db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:18:59.688363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbfd9c-8bd0-44cf-b168-11feb6e33db0 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:18:59.692096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aa5bf-d7dd-4106-8c04-a6b2991d943f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:18:59.692710 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aa5bf-d7dd-4106-8c04-a6b2991d943f service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:18:59.692990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aa5bf-d7dd-4106-8c04-a6b2991d943f service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:18:59.693277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aa5bf-d7dd-4106-8c04-a6b2991d943f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:18:59.722348 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:18:59.750956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394aa5bf-d7dd-4106-8c04-a6b2991d943f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:18:59.752399 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394aa5bf-d7dd-4106-8c04-a6b2991d943f service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:18:59.761790 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1359/2745] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:18:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:19:04.904349 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23ccfa3-22b4-4a83-b7df-9ff72ac7c296 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:04.968983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23ccfa3-22b4-4a83-b7df-9ff72ac7c296 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:04.973544 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d5cef-a2b0-4400-a73f-3f050911dac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:04.974192 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d5cef-a2b0-4400-a73f-3f050911dac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:04.974476 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d5cef-a2b0-4400-a73f-3f050911dac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:04.974748 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d5cef-a2b0-4400-a73f-3f050911dac3 admin admin] PATH_INFO: `/v3/users/6100e823d1c4464397c5e31a95e73e0a` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:04.975850 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d5cef-a2b0-4400-a73f-3f050911dac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6100e823d1c4464397c5e31a95e73e0a)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:04.977746 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d5cef-a2b0-4400-a73f-3f050911dac3 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:05.205948 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f80d5cef-a2b0-4400-a73f-3f050911dac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6100e823d1c4464397c5e31a95e73e0a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:05.219545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f80d5cef-a2b0-4400-a73f-3f050911dac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6100e823d1c4464397c5e31a95e73e0a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:05.226391 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f80d5cef-a2b0-4400-a73f-3f050911dac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6100e823d1c4464397c5e31a95e73e0a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:05.315810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f80d5cef-a2b0-4400-a73f-3f050911dac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6100e823d1c4464397c5e31a95e73e0a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:05.316430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f80d5cef-a2b0-4400-a73f-3f050911dac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6100e823d1c4464397c5e31a95e73e0a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:05.322792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1387/2746] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:19:04 2022] DELETE /identity/v3/users/6100e823d1c4464397c5e31a95e73e0a => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:19:05.328101 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133548d-cbd0-4417-8ea8-9eb62148bd48 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:05.337610 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133548d-cbd0-4417-8ea8-9eb62148bd48 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:05.337610 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133548d-cbd0-4417-8ea8-9eb62148bd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:05.349369 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1133548d-cbd0-4417-8ea8-9eb62148bd48 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:19:05.350639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1133548d-cbd0-4417-8ea8-9eb62148bd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:05.419675 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1360/2747] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:19:05.429480 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d5cef-a2b0-4400-a73f-3f050911dac3 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:05.430960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f80d5cef-a2b0-4400-a73f-3f050911dac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:05.459739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:19:05.498277 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d5cef-a2b0-4400-a73f-3f050911dac3 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:05.501986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234a0d5-26f9-4ff8-9d06-ee553dbe29fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:05.502609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234a0d5-26f9-4ff8-9d06-ee553dbe29fc service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:05.502903 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234a0d5-26f9-4ff8-9d06-ee553dbe29fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:05.503495 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234a0d5-26f9-4ff8-9d06-ee553dbe29fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:05.514072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234a0d5-26f9-4ff8-9d06-ee553dbe29fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:05.515547 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234a0d5-26f9-4ff8-9d06-ee553dbe29fc service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:05.525456 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1388/2748] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:19:05.957560 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133548d-cbd0-4417-8ea8-9eb62148bd48 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:05.959053 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1133548d-cbd0-4417-8ea8-9eb62148bd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:05.976499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133548d-cbd0-4417-8ea8-9eb62148bd48 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:05.982570 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43eba02-3f0c-4bd3-8458-5e6ebb8d00d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:05.983119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43eba02-3f0c-4bd3-8458-5e6ebb8d00d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:05.983387 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43eba02-3f0c-4bd3-8458-5e6ebb8d00d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:05.983695 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43eba02-3f0c-4bd3-8458-5e6ebb8d00d7 admin admin] PATH_INFO: `/v3/projects/9398e0d8a00a4a8ca5f10d8eb3521395` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:05.984820 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43eba02-3f0c-4bd3-8458-5e6ebb8d00d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9398e0d8a00a4a8ca5f10d8eb3521395)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:05.986794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43eba02-3f0c-4bd3-8458-5e6ebb8d00d7 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:06.098929 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d43eba02-3f0c-4bd3-8458-5e6ebb8d00d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9398e0d8a00a4a8ca5f10d8eb3521395', 'request_id': 'req-d43eba02-3f0c-4bd3-8458-5e6ebb8d00d7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:06.178554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d43eba02-3f0c-4bd3-8458-5e6ebb8d00d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9398e0d8a00a4a8ca5f10d8eb3521395', 'request_id': 'req-d43eba02-3f0c-4bd3-8458-5e6ebb8d00d7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:06.186952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d43eba02-3f0c-4bd3-8458-5e6ebb8d00d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9398e0d8a00a4a8ca5f10d8eb3521395', 'request_id': 'req-d43eba02-3f0c-4bd3-8458-5e6ebb8d00d7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:06.200393 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d43eba02-3f0c-4bd3-8458-5e6ebb8d00d7 admin admin] The token cache is being invalidate because project 9398e0d8a00a4a8ca5f10d8eb3521395 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:19:06.200691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d43eba02-3f0c-4bd3-8458-5e6ebb8d00d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:06.201357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d43eba02-3f0c-4bd3-8458-5e6ebb8d00d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:06.202908 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1361/2749] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:19:05 2022] DELETE /identity/v3/projects/9398e0d8a00a4a8ca5f10d8eb3521395 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:19:06.206870 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234a0d5-26f9-4ff8-9d06-ee553dbe29fc service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:06.217176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234a0d5-26f9-4ff8-9d06-ee553dbe29fc service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:06.275448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731c481-fa51-4175-9091-7c480423af78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:06.275996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731c481-fa51-4175-9091-7c480423af78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:06.276279 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731c481-fa51-4175-9091-7c480423af78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:06.276554 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731c481-fa51-4175-9091-7c480423af78 admin admin] PATH_INFO: `/v3/users/8ab26fd9a29c4869bc4a1f99d639a52e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:06.277758 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731c481-fa51-4175-9091-7c480423af78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ab26fd9a29c4869bc4a1f99d639a52e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:06.280374 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731c481-fa51-4175-9091-7c480423af78 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:06.461010 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d731c481-fa51-4175-9091-7c480423af78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ab26fd9a29c4869bc4a1f99d639a52e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:06.474318 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d731c481-fa51-4175-9091-7c480423af78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ab26fd9a29c4869bc4a1f99d639a52e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:06.481457 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d731c481-fa51-4175-9091-7c480423af78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ab26fd9a29c4869bc4a1f99d639a52e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:06.539273 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d731c481-fa51-4175-9091-7c480423af78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ab26fd9a29c4869bc4a1f99d639a52e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:06.539884 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d731c481-fa51-4175-9091-7c480423af78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ab26fd9a29c4869bc4a1f99d639a52e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:06.546698 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1389/2750] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:19:06 2022] DELETE /identity/v3/users/8ab26fd9a29c4869bc4a1f99d639a52e => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:19:07.043562 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43eba02-3f0c-4bd3-8458-5e6ebb8d00d7 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:07.045546 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d43eba02-3f0c-4bd3-8458-5e6ebb8d00d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:07.060641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43eba02-3f0c-4bd3-8458-5e6ebb8d00d7 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:07.112702 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:19:07.122030 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89940f96-773f-4a9b-bbb0-d41d036e3394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:07.123345 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89940f96-773f-4a9b-bbb0-d41d036e3394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:07.123639 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89940f96-773f-4a9b-bbb0-d41d036e3394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:07.123915 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89940f96-773f-4a9b-bbb0-d41d036e3394 admin admin] PATH_INFO: `/v3/projects/2e3fffbd8a6b437fb86ede09f52aa8c3` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:07.125055 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89940f96-773f-4a9b-bbb0-d41d036e3394 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e3fffbd8a6b437fb86ede09f52aa8c3)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:07.127025 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89940f96-773f-4a9b-bbb0-d41d036e3394 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:07.250328 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-89940f96-773f-4a9b-bbb0-d41d036e3394 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e3fffbd8a6b437fb86ede09f52aa8c3', 'request_id': 'req-89940f96-773f-4a9b-bbb0-d41d036e3394'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:07.367253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-89940f96-773f-4a9b-bbb0-d41d036e3394 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e3fffbd8a6b437fb86ede09f52aa8c3', 'request_id': 'req-89940f96-773f-4a9b-bbb0-d41d036e3394'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:07.373894 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-89940f96-773f-4a9b-bbb0-d41d036e3394 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e3fffbd8a6b437fb86ede09f52aa8c3', 'request_id': 'req-89940f96-773f-4a9b-bbb0-d41d036e3394'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:07.385438 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-89940f96-773f-4a9b-bbb0-d41d036e3394 admin admin] The token cache is being invalidate because project 2e3fffbd8a6b437fb86ede09f52aa8c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:19:07.385733 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-89940f96-773f-4a9b-bbb0-d41d036e3394 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:07.386431 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-89940f96-773f-4a9b-bbb0-d41d036e3394 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:07.387920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1362/2751] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:19:07 2022] DELETE /identity/v3/projects/2e3fffbd8a6b437fb86ede09f52aa8c3 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:19:07.392070 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731c481-fa51-4175-9091-7c480423af78 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:07.393552 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d731c481-fa51-4175-9091-7c480423af78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:07.407301 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731c481-fa51-4175-9091-7c480423af78 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:07.460906 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:19:07.474909 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2208fc2-deb5-4dda-86d5-09786e08ad44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:07.475823 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2208fc2-deb5-4dda-86d5-09786e08ad44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:07.476119 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2208fc2-deb5-4dda-86d5-09786e08ad44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:07.476400 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2208fc2-deb5-4dda-86d5-09786e08ad44 admin admin] PATH_INFO: `/v3/users/044fd434beab4f06acff4129ef561be8` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:07.477590 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2208fc2-deb5-4dda-86d5-09786e08ad44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=044fd434beab4f06acff4129ef561be8)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:07.479666 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2208fc2-deb5-4dda-86d5-09786e08ad44 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:07.767326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f2208fc2-deb5-4dda-86d5-09786e08ad44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '044fd434beab4f06acff4129ef561be8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:07.793453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f2208fc2-deb5-4dda-86d5-09786e08ad44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '044fd434beab4f06acff4129ef561be8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:07.803249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f2208fc2-deb5-4dda-86d5-09786e08ad44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '044fd434beab4f06acff4129ef561be8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:07.870451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f2208fc2-deb5-4dda-86d5-09786e08ad44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '044fd434beab4f06acff4129ef561be8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:07.873362 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f2208fc2-deb5-4dda-86d5-09786e08ad44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '044fd434beab4f06acff4129ef561be8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:07.881723 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1390/2752] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:19:07 2022] DELETE /identity/v3/users/044fd434beab4f06acff4129ef561be8 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:19:08.381089 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89940f96-773f-4a9b-bbb0-d41d036e3394 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:08.383309 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-89940f96-773f-4a9b-bbb0-d41d036e3394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:08.402229 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89940f96-773f-4a9b-bbb0-d41d036e3394 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:08.466575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820d9ba-02b6-447d-8d10-7f1c5829fb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:08.467215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820d9ba-02b6-447d-8d10-7f1c5829fb31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:08.467507 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820d9ba-02b6-447d-8d10-7f1c5829fb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:08.467791 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820d9ba-02b6-447d-8d10-7f1c5829fb31 admin admin] PATH_INFO: `/v3/projects/31b08f8253c049d7875660581f405d14` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:08.468945 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820d9ba-02b6-447d-8d10-7f1c5829fb31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31b08f8253c049d7875660581f405d14)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:08.471092 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820d9ba-02b6-447d-8d10-7f1c5829fb31 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:08.552602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2208fc2-deb5-4dda-86d5-09786e08ad44 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:08.554203 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f2208fc2-deb5-4dda-86d5-09786e08ad44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:08.613259 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2208fc2-deb5-4dda-86d5-09786e08ad44 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:08.626598 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5820d9ba-02b6-447d-8d10-7f1c5829fb31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31b08f8253c049d7875660581f405d14', 'request_id': 'req-5820d9ba-02b6-447d-8d10-7f1c5829fb31'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:08.663550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817eda6-407f-492f-a286-12aa3943878f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:08.664163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817eda6-407f-492f-a286-12aa3943878f service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:08.664448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817eda6-407f-492f-a286-12aa3943878f service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:08.664727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817eda6-407f-492f-a286-12aa3943878f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:08.666799 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9817eda6-407f-492f-a286-12aa3943878f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:08.677153 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817eda6-407f-492f-a286-12aa3943878f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:08.678902 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817eda6-407f-492f-a286-12aa3943878f service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:08.687606 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1391/2753] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:19:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:19:08.704393 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5820d9ba-02b6-447d-8d10-7f1c5829fb31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31b08f8253c049d7875660581f405d14', 'request_id': 'req-5820d9ba-02b6-447d-8d10-7f1c5829fb31'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:08.711056 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5820d9ba-02b6-447d-8d10-7f1c5829fb31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31b08f8253c049d7875660581f405d14', 'request_id': 'req-5820d9ba-02b6-447d-8d10-7f1c5829fb31'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:08.723325 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5820d9ba-02b6-447d-8d10-7f1c5829fb31 admin admin] The token cache is being invalidate because project 31b08f8253c049d7875660581f405d14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:19:08.723614 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5820d9ba-02b6-447d-8d10-7f1c5829fb31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:08.724276 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5820d9ba-02b6-447d-8d10-7f1c5829fb31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:08.725835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1363/2754] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:19:08 2022] DELETE /identity/v3/projects/31b08f8253c049d7875660581f405d14 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:19:08.731879 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8cdb12-8384-4f0a-b94c-ab5846867792 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:08.732169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8cdb12-8384-4f0a-b94c-ab5846867792 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:08.732449 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8cdb12-8384-4f0a-b94c-ab5846867792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:08.752258 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2c8cdb12-8384-4f0a-b94c-ab5846867792 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:19:08.753480 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2c8cdb12-8384-4f0a-b94c-ab5846867792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:08.774810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:19:08.823568 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1392/2755] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:19:08.832818 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb69a6-45b8-476c-94a8-5e134966cd66 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:08.833107 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb69a6-45b8-476c-94a8-5e134966cd66 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:08.833391 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb69a6-45b8-476c-94a8-5e134966cd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:08.854237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-26fb69a6-45b8-476c-94a8-5e134966cd66 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:19:08.855495 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-26fb69a6-45b8-476c-94a8-5e134966cd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:08.863919 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1364/2756] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:19:08.868068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8cdb12-8384-4f0a-b94c-ab5846867792 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:08.879921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8cdb12-8384-4f0a-b94c-ab5846867792 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:08.884903 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2c7ab-e4f0-4f2e-99ad-3eae263d573d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:08.885545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2c7ab-e4f0-4f2e-99ad-3eae263d573d admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:08.885829 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2c7ab-e4f0-4f2e-99ad-3eae263d573d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:08.886519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2c7ab-e4f0-4f2e-99ad-3eae263d573d admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:08.886777 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2c7ab-e4f0-4f2e-99ad-3eae263d573d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:19:08.887083 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2c7ab-e4f0-4f2e-99ad-3eae263d573d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:08.889606 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2c7ab-e4f0-4f2e-99ad-3eae263d573d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:08.902599 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1393/2757] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:19:08.907164 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb69a6-45b8-476c-94a8-5e134966cd66 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:08.917455 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb69a6-45b8-476c-94a8-5e134966cd66 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:08.922941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9de4db-d670-4cd4-a27d-f5792105233a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:08.923569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9de4db-d670-4cd4-a27d-f5792105233a admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:08.923854 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9de4db-d670-4cd4-a27d-f5792105233a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:08.924130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9de4db-d670-4cd4-a27d-f5792105233a admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:08.924788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9de4db-d670-4cd4-a27d-f5792105233a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:08.926866 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9de4db-d670-4cd4-a27d-f5792105233a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:08.931764 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:19:09.046393 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1365/2758] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:19:08 2022] POST /identity/v3/projects => generated 359 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:19:09.047595 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2c7ab-e4f0-4f2e-99ad-3eae263d573d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:09.059295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2c7ab-e4f0-4f2e-99ad-3eae263d573d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:09.121053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b39c6-472e-477c-bf19-ce463276c4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:09.121572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b39c6-472e-477c-bf19-ce463276c4ea admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:09.121892 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b39c6-472e-477c-bf19-ce463276c4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:09.122704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b39c6-472e-477c-bf19-ce463276c4ea admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:09.123358 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b39c6-472e-477c-bf19-ce463276c4ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:09.125275 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b39c6-472e-477c-bf19-ce463276c4ea admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:09.228080 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1394/2759] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:19:09 2022] POST /identity/v3/users => generated 280 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:19:09.229276 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9de4db-d670-4cd4-a27d-f5792105233a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:09.241151 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9de4db-d670-4cd4-a27d-f5792105233a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:09.246149 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecc397-1694-4215-b3ef-3076fd8cf3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:09.246852 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecc397-1694-4215-b3ef-3076fd8cf3e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:09.247839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecc397-1694-4215-b3ef-3076fd8cf3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:09.248197 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecc397-1694-4215-b3ef-3076fd8cf3e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:09.248911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeecc397-1694-4215-b3ef-3076fd8cf3e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:09.250604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeecc397-1694-4215-b3ef-3076fd8cf3e2 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:09.260693 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1366/2760] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:19:09.264833 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b39c6-472e-477c-bf19-ce463276c4ea admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:09.275056 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b39c6-472e-477c-bf19-ce463276c4ea admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:09.279782 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6adde-ffa4-4bc7-bc62-c3a48790779f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:09.280378 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6adde-ffa4-4bc7-bc62-c3a48790779f admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:09.280658 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6adde-ffa4-4bc7-bc62-c3a48790779f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:09.280929 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6adde-ffa4-4bc7-bc62-c3a48790779f admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:09.281565 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6adde-ffa4-4bc7-bc62-c3a48790779f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:09.283178 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6adde-ffa4-4bc7-bc62-c3a48790779f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:09.296140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1395/2761] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:19:09.300516 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecc397-1694-4215-b3ef-3076fd8cf3e2 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:09.311494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecc397-1694-4215-b3ef-3076fd8cf3e2 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:09.316129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a7525-2dd1-418e-8569-13e2c257dea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:09.316822 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a7525-2dd1-418e-8569-13e2c257dea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:09.317104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a7525-2dd1-418e-8569-13e2c257dea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:09.317378 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a7525-2dd1-418e-8569-13e2c257dea6 admin admin] PATH_INFO: `/v3/projects/aa9401aa2e494ac3b4057e54a01561e3/users/402d38a5e98843f4b71cb63a7d5332f0/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:09.333140 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a7525-2dd1-418e-8569-13e2c257dea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa9401aa2e494ac3b4057e54a01561e3, user_id=402d38a5e98843f4b71cb63a7d5332f0, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:09.335940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a7525-2dd1-418e-8569-13e2c257dea6 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:09.470780 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1367/2762] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:19:09 2022] PUT /identity/v3/projects/aa9401aa2e494ac3b4057e54a01561e3/users/402d38a5e98843f4b71cb63a7d5332f0/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:19:09.539140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f15a71-ba6d-451e-9ff7-7006b33d2681 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:09.539427 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f15a71-ba6d-451e-9ff7-7006b33d2681 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:09.539704 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f15a71-ba6d-451e-9ff7-7006b33d2681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:09.567916 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d4f15a71-ba6d-451e-9ff7-7006b33d2681 None None] MFA Rules not processed for user `402d38a5e98843f4b71cb63a7d5332f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:19:09.569137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d4f15a71-ba6d-451e-9ff7-7006b33d2681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:09.636006 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1396/2763] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:19:09 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:19:09.671681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a7525-2dd1-418e-8569-13e2c257dea6 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:09.673426 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b62a7525-2dd1-418e-8569-13e2c257dea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:09.737906 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a7525-2dd1-418e-8569-13e2c257dea6 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:09.741351 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8dc5f-2023-4a49-af4d-e9234b1a6bc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:09.741993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8dc5f-2023-4a49-af4d-e9234b1a6bc7 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:09.742281 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8dc5f-2023-4a49-af4d-e9234b1a6bc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:09.742562 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8dc5f-2023-4a49-af4d-e9234b1a6bc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:09.750849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8dc5f-2023-4a49-af4d-e9234b1a6bc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:09.752317 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8dc5f-2023-4a49-af4d-e9234b1a6bc7 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:09.761090 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1368/2764] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:19:09 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:19:09.776693 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f15a71-ba6d-451e-9ff7-7006b33d2681 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:09.798835 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:19:09.841373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f15a71-ba6d-451e-9ff7-7006b33d2681 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:09.845988 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa87cb7-95de-4a50-8030-04d54763e9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:09.846617 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa87cb7-95de-4a50-8030-04d54763e9e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:09.846895 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa87cb7-95de-4a50-8030-04d54763e9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:09.847165 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa87cb7-95de-4a50-8030-04d54763e9e5 admin admin] PATH_INFO: `/v3/users/402d38a5e98843f4b71cb63a7d5332f0` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:09.848285 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa87cb7-95de-4a50-8030-04d54763e9e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=402d38a5e98843f4b71cb63a7d5332f0)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:09.850249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa87cb7-95de-4a50-8030-04d54763e9e5 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:10.016416 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6aa87cb7-95de-4a50-8030-04d54763e9e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '402d38a5e98843f4b71cb63a7d5332f0'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:10.029556 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6aa87cb7-95de-4a50-8030-04d54763e9e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '402d38a5e98843f4b71cb63a7d5332f0'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:10.036364 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6aa87cb7-95de-4a50-8030-04d54763e9e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '402d38a5e98843f4b71cb63a7d5332f0'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:10.098618 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6aa87cb7-95de-4a50-8030-04d54763e9e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '402d38a5e98843f4b71cb63a7d5332f0'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:10.099238 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6aa87cb7-95de-4a50-8030-04d54763e9e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '402d38a5e98843f4b71cb63a7d5332f0'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:10.105519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1397/2765] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:19:09 2022] DELETE /identity/v3/users/402d38a5e98843f4b71cb63a7d5332f0 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:19:10.112452 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd2c3f-e7e3-4869-9fed-c29af267b9d3 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:10.112452 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd2c3f-e7e3-4869-9fed-c29af267b9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:10.112452 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd2c3f-e7e3-4869-9fed-c29af267b9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:10.130262 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-37fd2c3f-e7e3-4869-9fed-c29af267b9d3 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:19:10.131539 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-37fd2c3f-e7e3-4869-9fed-c29af267b9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:10.192575 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1369/2766] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:19:10.202361 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa87cb7-95de-4a50-8030-04d54763e9e5 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:10.203820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6aa87cb7-95de-4a50-8030-04d54763e9e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:10.261087 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa87cb7-95de-4a50-8030-04d54763e9e5 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:10.264980 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61272f59-5a6b-4ea5-ac2a-98752933ee22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:10.265604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61272f59-5a6b-4ea5-ac2a-98752933ee22 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:10.265917 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61272f59-5a6b-4ea5-ac2a-98752933ee22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:10.266199 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61272f59-5a6b-4ea5-ac2a-98752933ee22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:10.276497 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61272f59-5a6b-4ea5-ac2a-98752933ee22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:10.277874 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61272f59-5a6b-4ea5-ac2a-98752933ee22 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:10.287458 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1398/2767] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:19:10.763484 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd2c3f-e7e3-4869-9fed-c29af267b9d3 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:10.766146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-37fd2c3f-e7e3-4869-9fed-c29af267b9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:10.777804 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd2c3f-e7e3-4869-9fed-c29af267b9d3 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:10.783142 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27052fd2-59a4-4858-85ac-472a76a2bf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:10.783831 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27052fd2-59a4-4858-85ac-472a76a2bf25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:10.784116 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27052fd2-59a4-4858-85ac-472a76a2bf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:10.784432 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27052fd2-59a4-4858-85ac-472a76a2bf25 admin admin] PATH_INFO: `/v3/projects/aa9401aa2e494ac3b4057e54a01561e3` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:10.785592 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27052fd2-59a4-4858-85ac-472a76a2bf25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa9401aa2e494ac3b4057e54a01561e3)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:10.787569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27052fd2-59a4-4858-85ac-472a76a2bf25 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:10.912358 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:19:10.913778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-27052fd2-59a4-4858-85ac-472a76a2bf25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa9401aa2e494ac3b4057e54a01561e3', 'request_id': 'req-27052fd2-59a4-4858-85ac-472a76a2bf25'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:10.995723 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-27052fd2-59a4-4858-85ac-472a76a2bf25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa9401aa2e494ac3b4057e54a01561e3', 'request_id': 'req-27052fd2-59a4-4858-85ac-472a76a2bf25'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:11.001828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-27052fd2-59a4-4858-85ac-472a76a2bf25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa9401aa2e494ac3b4057e54a01561e3', 'request_id': 'req-27052fd2-59a4-4858-85ac-472a76a2bf25'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:11.013996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-27052fd2-59a4-4858-85ac-472a76a2bf25 admin admin] The token cache is being invalidate because project aa9401aa2e494ac3b4057e54a01561e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:19:11.014317 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-27052fd2-59a4-4858-85ac-472a76a2bf25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:11.014959 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-27052fd2-59a4-4858-85ac-472a76a2bf25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:11.016394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1370/2768] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:19:10 2022] DELETE /identity/v3/projects/aa9401aa2e494ac3b4057e54a01561e3 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:19:11.021276 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead74cc-9ad4-44ac-b81b-a864237edbb3 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:11.022057 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead74cc-9ad4-44ac-b81b-a864237edbb3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:11.022142 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead74cc-9ad4-44ac-b81b-a864237edbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:11.043440 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8ead74cc-9ad4-44ac-b81b-a864237edbb3 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:19:11.044567 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8ead74cc-9ad4-44ac-b81b-a864237edbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:11.081767 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:19:11.104972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1399/2769] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:19:11.113699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90809897-caa3-495f-8c0b-fc851ae9ad53 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:11.114411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90809897-caa3-495f-8c0b-fc851ae9ad53 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:11.114922 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90809897-caa3-495f-8c0b-fc851ae9ad53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:11.137424 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-90809897-caa3-495f-8c0b-fc851ae9ad53 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:19:11.139307 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-90809897-caa3-495f-8c0b-fc851ae9ad53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:11.147908 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1371/2770] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:19:11.152182 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead74cc-9ad4-44ac-b81b-a864237edbb3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:11.162382 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead74cc-9ad4-44ac-b81b-a864237edbb3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:11.169377 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e40c5-1b35-40c0-8004-ba2ab64ba388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:11.169377 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e40c5-1b35-40c0-8004-ba2ab64ba388 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:11.169377 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e40c5-1b35-40c0-8004-ba2ab64ba388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:11.169377 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e40c5-1b35-40c0-8004-ba2ab64ba388 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:11.169787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e40c5-1b35-40c0-8004-ba2ab64ba388 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:19:11.169787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e40c5-1b35-40c0-8004-ba2ab64ba388 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:11.170810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e40c5-1b35-40c0-8004-ba2ab64ba388 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:11.182239 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1400/2771] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:19:11.187334 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90809897-caa3-495f-8c0b-fc851ae9ad53 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:11.196469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90809897-caa3-495f-8c0b-fc851ae9ad53 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:11.201775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1007417-df09-4187-b396-d24cd9df6153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:11.202427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1007417-df09-4187-b396-d24cd9df6153 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:11.202710 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1007417-df09-4187-b396-d24cd9df6153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:11.202983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1007417-df09-4187-b396-d24cd9df6153 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:11.203629 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1007417-df09-4187-b396-d24cd9df6153 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:11.205566 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1007417-df09-4187-b396-d24cd9df6153 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:11.357917 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1372/2772] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:19:11 2022] POST /identity/v3/projects => generated 371 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:19:11.361505 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e40c5-1b35-40c0-8004-ba2ab64ba388 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:11.373088 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e40c5-1b35-40c0-8004-ba2ab64ba388 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:11.434084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52d43e-39ae-44f7-9e09-2e5957775c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:11.435202 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52d43e-39ae-44f7-9e09-2e5957775c9c admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:11.435610 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52d43e-39ae-44f7-9e09-2e5957775c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:11.435790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52d43e-39ae-44f7-9e09-2e5957775c9c admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:11.436533 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52d43e-39ae-44f7-9e09-2e5957775c9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:11.438362 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52d43e-39ae-44f7-9e09-2e5957775c9c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:11.540548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1401/2773] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:19:11 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:19:11.545153 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1007417-df09-4187-b396-d24cd9df6153 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:11.555980 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1007417-df09-4187-b396-d24cd9df6153 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:11.561260 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd31385-9c00-4d95-9337-29ec7b95a75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:11.561879 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd31385-9c00-4d95-9337-29ec7b95a75d admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:11.562196 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd31385-9c00-4d95-9337-29ec7b95a75d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:11.562583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd31385-9c00-4d95-9337-29ec7b95a75d admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:11.563236 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd31385-9c00-4d95-9337-29ec7b95a75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:11.564849 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd31385-9c00-4d95-9337-29ec7b95a75d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:11.578582 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1373/2774] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:19:11.587331 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52d43e-39ae-44f7-9e09-2e5957775c9c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:11.590674 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52d43e-39ae-44f7-9e09-2e5957775c9c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:11.595639 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5aad90-97d1-4b46-a7f4-66b73e7807af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:11.596260 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5aad90-97d1-4b46-a7f4-66b73e7807af admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:11.596542 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5aad90-97d1-4b46-a7f4-66b73e7807af admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:11.596819 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5aad90-97d1-4b46-a7f4-66b73e7807af admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:11.597470 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5aad90-97d1-4b46-a7f4-66b73e7807af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:11.599094 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5aad90-97d1-4b46-a7f4-66b73e7807af admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:11.611217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1402/2775] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:19:11.615689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd31385-9c00-4d95-9337-29ec7b95a75d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:11.626644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd31385-9c00-4d95-9337-29ec7b95a75d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:11.631561 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8236674-be95-4815-b053-7b0664a56464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:11.632227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8236674-be95-4815-b053-7b0664a56464 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:11.645156 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8236674-be95-4815-b053-7b0664a56464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:11.645156 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8236674-be95-4815-b053-7b0664a56464 admin admin] PATH_INFO: `/v3/projects/a0a7b8230f084ee6a22891b47aae36c4/users/6b0deeac7e344ffd8ae26d01595d2225/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:11.651279 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8236674-be95-4815-b053-7b0664a56464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a7b8230f084ee6a22891b47aae36c4, user_id=6b0deeac7e344ffd8ae26d01595d2225, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:11.654021 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8236674-be95-4815-b053-7b0664a56464 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:11.784593 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1374/2776] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:19:11 2022] PUT /identity/v3/projects/a0a7b8230f084ee6a22891b47aae36c4/users/6b0deeac7e344ffd8ae26d01595d2225/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:19:11.795564 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd4279-230c-4fb6-adc9-096e731c22a2 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:11.795853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd4279-230c-4fb6-adc9-096e731c22a2 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:11.796129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd4279-230c-4fb6-adc9-096e731c22a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:11.827525 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-5bcd4279-230c-4fb6-adc9-096e731c22a2 None None] MFA Rules not processed for user `6b0deeac7e344ffd8ae26d01595d2225`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:19:11.828870 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5bcd4279-230c-4fb6-adc9-096e731c22a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:11.895880 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1403/2777] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:19:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:19:11.908143 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8236674-be95-4815-b053-7b0664a56464 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:11.910075 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a8236674-be95-4815-b053-7b0664a56464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:11.971476 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8236674-be95-4815-b053-7b0664a56464 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:11.975149 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08830a6-ba1a-477b-b92d-5318f596694d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:11.975766 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08830a6-ba1a-477b-b92d-5318f596694d service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:11.976051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08830a6-ba1a-477b-b92d-5318f596694d service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:11.976328 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08830a6-ba1a-477b-b92d-5318f596694d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:11.985478 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08830a6-ba1a-477b-b92d-5318f596694d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:11.986942 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08830a6-ba1a-477b-b92d-5318f596694d service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:11.996071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1375/2778] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:19:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:19:17.076712 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd4279-230c-4fb6-adc9-096e731c22a2 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:17.119981 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:19:17.142940 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd4279-230c-4fb6-adc9-096e731c22a2 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:17.147697 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb866f-c910-436f-b3d8-14851105e150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:17.148335 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb866f-c910-436f-b3d8-14851105e150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:17.148613 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb866f-c910-436f-b3d8-14851105e150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:17.148886 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb866f-c910-436f-b3d8-14851105e150 admin admin] PATH_INFO: `/v3/users/6b0deeac7e344ffd8ae26d01595d2225` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:17.150041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb866f-c910-436f-b3d8-14851105e150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b0deeac7e344ffd8ae26d01595d2225)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:17.151992 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb866f-c910-436f-b3d8-14851105e150 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:17.346794 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c8eb866f-c910-436f-b3d8-14851105e150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b0deeac7e344ffd8ae26d01595d2225'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:17.359172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c8eb866f-c910-436f-b3d8-14851105e150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b0deeac7e344ffd8ae26d01595d2225'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:17.366101 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c8eb866f-c910-436f-b3d8-14851105e150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b0deeac7e344ffd8ae26d01595d2225'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:17.429954 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c8eb866f-c910-436f-b3d8-14851105e150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b0deeac7e344ffd8ae26d01595d2225'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:17.430561 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c8eb866f-c910-436f-b3d8-14851105e150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b0deeac7e344ffd8ae26d01595d2225'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:17.437712 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1404/2779] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:19:17 2022] DELETE /identity/v3/users/6b0deeac7e344ffd8ae26d01595d2225 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:19:17.442239 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6bf67-1f86-4c49-af2e-f7228338c820 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:17.442737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6bf67-1f86-4c49-af2e-f7228338c820 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:17.442838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6bf67-1f86-4c49-af2e-f7228338c820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:17.462486 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f7b6bf67-1f86-4c49-af2e-f7228338c820 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:19:17.463855 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f7b6bf67-1f86-4c49-af2e-f7228338c820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:17.526962 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1376/2780] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:19:17.536720 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb866f-c910-436f-b3d8-14851105e150 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:17.538984 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c8eb866f-c910-436f-b3d8-14851105e150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:17.610506 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb866f-c910-436f-b3d8-14851105e150 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:17.615097 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2327a-37bb-43f7-958e-c0201e0ebd6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:17.620774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2327a-37bb-43f7-958e-c0201e0ebd6d service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:17.620774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2327a-37bb-43f7-958e-c0201e0ebd6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:17.620774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2327a-37bb-43f7-958e-c0201e0ebd6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:17.626416 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2327a-37bb-43f7-958e-c0201e0ebd6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:17.627846 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2327a-37bb-43f7-958e-c0201e0ebd6d service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:17.637996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1405/2781] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:19:18.158130 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6bf67-1f86-4c49-af2e-f7228338c820 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:18.159698 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f7b6bf67-1f86-4c49-af2e-f7228338c820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:18.172626 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6bf67-1f86-4c49-af2e-f7228338c820 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:18.177821 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a67fc-55d5-4241-98df-2f70de2897d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:18.178534 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a67fc-55d5-4241-98df-2f70de2897d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:18.178827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a67fc-55d5-4241-98df-2f70de2897d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:18.179269 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a67fc-55d5-4241-98df-2f70de2897d8 admin admin] PATH_INFO: `/v3/projects/a0a7b8230f084ee6a22891b47aae36c4` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:18.180247 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a67fc-55d5-4241-98df-2f70de2897d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a7b8230f084ee6a22891b47aae36c4)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:18.182329 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a67fc-55d5-4241-98df-2f70de2897d8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:18.194231 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:19:18.328763 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2f1a67fc-55d5-4241-98df-2f70de2897d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a7b8230f084ee6a22891b47aae36c4', 'request_id': 'req-2f1a67fc-55d5-4241-98df-2f70de2897d8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:18.436042 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2f1a67fc-55d5-4241-98df-2f70de2897d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a7b8230f084ee6a22891b47aae36c4', 'request_id': 'req-2f1a67fc-55d5-4241-98df-2f70de2897d8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:18.442692 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2f1a67fc-55d5-4241-98df-2f70de2897d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a7b8230f084ee6a22891b47aae36c4', 'request_id': 'req-2f1a67fc-55d5-4241-98df-2f70de2897d8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:18.455912 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2f1a67fc-55d5-4241-98df-2f70de2897d8 admin admin] The token cache is being invalidate because project a0a7b8230f084ee6a22891b47aae36c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:19:18.456259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2f1a67fc-55d5-4241-98df-2f70de2897d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:18.456993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2f1a67fc-55d5-4241-98df-2f70de2897d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:19:18.458740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1377/2782] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:19:18 2022] DELETE /identity/v3/projects/a0a7b8230f084ee6a22891b47aae36c4 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:19:18.463989 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76225fb7-dc49-490c-b0ec-fbeae6912021 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:18.464507 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76225fb7-dc49-490c-b0ec-fbeae6912021 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:18.464613 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76225fb7-dc49-490c-b0ec-fbeae6912021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:18.484904 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-76225fb7-dc49-490c-b0ec-fbeae6912021 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:19:18.486215 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-76225fb7-dc49-490c-b0ec-fbeae6912021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:18.539430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:19:18.548035 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1406/2783] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:19:18.556125 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b65014-6c52-400b-8dd9-eb3e573ee1e9 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:18.556413 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b65014-6c52-400b-8dd9-eb3e573ee1e9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:18.556689 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b65014-6c52-400b-8dd9-eb3e573ee1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:18.579985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-03b65014-6c52-400b-8dd9-eb3e573ee1e9 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:19:18.581229 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-03b65014-6c52-400b-8dd9-eb3e573ee1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:18.589256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1378/2784] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:19:18.594231 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76225fb7-dc49-490c-b0ec-fbeae6912021 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:18.604421 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76225fb7-dc49-490c-b0ec-fbeae6912021 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:18.608991 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec66db-8add-4b7a-8112-953bd3334184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:18.609586 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec66db-8add-4b7a-8112-953bd3334184 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:18.610926 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec66db-8add-4b7a-8112-953bd3334184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:18.610926 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec66db-8add-4b7a-8112-953bd3334184 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:18.610926 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec66db-8add-4b7a-8112-953bd3334184 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:19:18.611243 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec66db-8add-4b7a-8112-953bd3334184 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:18.612833 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec66db-8add-4b7a-8112-953bd3334184 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:18.624913 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1407/2785] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:19:18.629503 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b65014-6c52-400b-8dd9-eb3e573ee1e9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:18.639686 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b65014-6c52-400b-8dd9-eb3e573ee1e9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:18.644410 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f6c94-4aee-4659-b2dc-c0f4d92e9123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:18.645505 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f6c94-4aee-4659-b2dc-c0f4d92e9123 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:18.645505 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f6c94-4aee-4659-b2dc-c0f4d92e9123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:18.645771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f6c94-4aee-4659-b2dc-c0f4d92e9123 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:18.646205 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f6c94-4aee-4659-b2dc-c0f4d92e9123 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:18.648382 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f6c94-4aee-4659-b2dc-c0f4d92e9123 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:18.734317 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1379/2786] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:19:18 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:19:18.738472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec66db-8add-4b7a-8112-953bd3334184 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:18.749698 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec66db-8add-4b7a-8112-953bd3334184 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:18.812944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d5979-9aff-48e0-8263-51e9656082da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:18.813570 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d5979-9aff-48e0-8263-51e9656082da admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:18.814188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d5979-9aff-48e0-8263-51e9656082da admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:18.814472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d5979-9aff-48e0-8263-51e9656082da admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:18.815176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d5979-9aff-48e0-8263-51e9656082da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:18.817177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d5979-9aff-48e0-8263-51e9656082da admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:18.916876 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1408/2787] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:19:18 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:19:18.921733 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f6c94-4aee-4659-b2dc-c0f4d92e9123 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:18.932783 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f6c94-4aee-4659-b2dc-c0f4d92e9123 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:18.938194 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faaac9c-7ad9-4e10-b1df-21118bbfb7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:18.938803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faaac9c-7ad9-4e10-b1df-21118bbfb7fd admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:18.939089 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faaac9c-7ad9-4e10-b1df-21118bbfb7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:18.939368 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faaac9c-7ad9-4e10-b1df-21118bbfb7fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:18.940013 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faaac9c-7ad9-4e10-b1df-21118bbfb7fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:18.941623 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faaac9c-7ad9-4e10-b1df-21118bbfb7fd admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:18.955034 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1380/2788] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:19:18.957898 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d5979-9aff-48e0-8263-51e9656082da admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:18.968455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d5979-9aff-48e0-8263-51e9656082da admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:18.973268 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357dbaf-fab5-4537-a0cf-80607bbeee68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:18.973891 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357dbaf-fab5-4537-a0cf-80607bbeee68 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:18.974184 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357dbaf-fab5-4537-a0cf-80607bbeee68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:18.974455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357dbaf-fab5-4537-a0cf-80607bbeee68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:18.975093 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357dbaf-fab5-4537-a0cf-80607bbeee68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:18.976694 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357dbaf-fab5-4537-a0cf-80607bbeee68 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:18.989834 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1409/2789] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:19:18.994278 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faaac9c-7ad9-4e10-b1df-21118bbfb7fd admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:19.006451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faaac9c-7ad9-4e10-b1df-21118bbfb7fd admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:19.011430 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046421f-976b-4b12-8deb-09db7dd87e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:19.012457 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046421f-976b-4b12-8deb-09db7dd87e62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:19.012669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046421f-976b-4b12-8deb-09db7dd87e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:19.012965 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046421f-976b-4b12-8deb-09db7dd87e62 admin admin] PATH_INFO: `/v3/projects/e0cdab1c080d4132adc6cd64d8627c68/users/33f480ef1b8b49df83276a66f445ae85/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:19.028743 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8046421f-976b-4b12-8deb-09db7dd87e62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0cdab1c080d4132adc6cd64d8627c68, user_id=33f480ef1b8b49df83276a66f445ae85, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:19.031469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8046421f-976b-4b12-8deb-09db7dd87e62 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:19.191411 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1381/2790] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:19:18 2022] PUT /identity/v3/projects/e0cdab1c080d4132adc6cd64d8627c68/users/33f480ef1b8b49df83276a66f445ae85/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:19:19.197169 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5575b9-ccc4-427f-b966-9a1a36543b28 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:19.197531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5575b9-ccc4-427f-b966-9a1a36543b28 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:19.197902 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5575b9-ccc4-427f-b966-9a1a36543b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:19.217170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1f5575b9-ccc4-427f-b966-9a1a36543b28 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:19:19.218340 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1f5575b9-ccc4-427f-b966-9a1a36543b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:19.278300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1410/2791] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:19:19.287903 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046421f-976b-4b12-8deb-09db7dd87e62 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:19.289618 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8046421f-976b-4b12-8deb-09db7dd87e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:19.347827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046421f-976b-4b12-8deb-09db7dd87e62 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:19.351609 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd76413-b9ed-4946-9831-261412be5f7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:19.352214 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd76413-b9ed-4946-9831-261412be5f7f service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:19.352501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd76413-b9ed-4946-9831-261412be5f7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:19.352783 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd76413-b9ed-4946-9831-261412be5f7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:19.362565 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd76413-b9ed-4946-9831-261412be5f7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:19.364004 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd76413-b9ed-4946-9831-261412be5f7f service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:19.373838 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1382/2792] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:19:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:19:25.093305 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51998833-3d50-4d3b-b663-e74b9da277bc None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:25.093827 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51998833-3d50-4d3b-b663-e74b9da277bc None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:25.093827 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51998833-3d50-4d3b-b663-e74b9da277bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:25.133622 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-51998833-3d50-4d3b-b663-e74b9da277bc None None] MFA Rules not processed for user `33f480ef1b8b49df83276a66f445ae85`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:19:25.134817 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-51998833-3d50-4d3b-b663-e74b9da277bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:25.217222 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:19:25.235472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1411/2793] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:19:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:19:25.241764 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd76413-b9ed-4946-9831-261412be5f7f service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:25.256928 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd76413-b9ed-4946-9831-261412be5f7f service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:25.268017 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa1f2a-20fb-4423-9497-c98ec54505c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:25.268892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa1f2a-20fb-4423-9497-c98ec54505c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:25.270065 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa1f2a-20fb-4423-9497-c98ec54505c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:25.270065 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa1f2a-20fb-4423-9497-c98ec54505c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:25.270902 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa1f2a-20fb-4423-9497-c98ec54505c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:25.273109 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa1f2a-20fb-4423-9497-c98ec54505c3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:25.371270 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1383/2794] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:19:25 2022] POST /identity/v3/projects => generated 379 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:19:25.375643 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51998833-3d50-4d3b-b663-e74b9da277bc None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:25.387997 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51998833-3d50-4d3b-b663-e74b9da277bc None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:25.472584 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b22e8c-e90c-46f7-875c-7ac4900237ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:25.473383 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b22e8c-e90c-46f7-875c-7ac4900237ea admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:25.473885 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b22e8c-e90c-46f7-875c-7ac4900237ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:25.474381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b22e8c-e90c-46f7-875c-7ac4900237ea admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:25.476688 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b22e8c-e90c-46f7-875c-7ac4900237ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:25.481158 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b22e8c-e90c-46f7-875c-7ac4900237ea admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:25.564052 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa1f2a-20fb-4423-9497-c98ec54505c3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:25.565508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1412/2795] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:19:25 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:19:25.575421 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa1f2a-20fb-4423-9497-c98ec54505c3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:25.586375 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414b6dd-ca60-4fa9-a39b-e5bd6733f8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:25.587571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414b6dd-ca60-4fa9-a39b-e5bd6733f8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:25.587865 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414b6dd-ca60-4fa9-a39b-e5bd6733f8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:25.588147 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414b6dd-ca60-4fa9-a39b-e5bd6733f8a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:25.588827 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414b6dd-ca60-4fa9-a39b-e5bd6733f8a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:25.590549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414b6dd-ca60-4fa9-a39b-e5bd6733f8a5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:25.603310 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1384/2796] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:19:25.608601 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b22e8c-e90c-46f7-875c-7ac4900237ea admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:25.625718 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b22e8c-e90c-46f7-875c-7ac4900237ea admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:25.634975 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938854aa-52be-4240-918c-52e1d3376824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:25.635688 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938854aa-52be-4240-918c-52e1d3376824 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:25.635995 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938854aa-52be-4240-918c-52e1d3376824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:25.636242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938854aa-52be-4240-918c-52e1d3376824 admin admin] PATH_INFO: `/v3/projects/c7041a5e4c714f5fa5b77e8b2c3af41b/users/2458d329b68d4a44814293fb42fe0eda/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:25.658750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938854aa-52be-4240-918c-52e1d3376824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7041a5e4c714f5fa5b77e8b2c3af41b, user_id=2458d329b68d4a44814293fb42fe0eda, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:25.662895 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938854aa-52be-4240-918c-52e1d3376824 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:25.730191 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1413/2797] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:19:25 2022] PUT /identity/v3/projects/c7041a5e4c714f5fa5b77e8b2c3af41b/users/2458d329b68d4a44814293fb42fe0eda/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:19:25.735129 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414b6dd-ca60-4fa9-a39b-e5bd6733f8a5 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:25.742740 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:19:25.747137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414b6dd-ca60-4fa9-a39b-e5bd6733f8a5 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:25.818851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51aeb22-ecfb-4f25-9c4e-0c4f1ba30339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:25.819507 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aeb22-ecfb-4f25-9c4e-0c4f1ba30339 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:25.819852 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aeb22-ecfb-4f25-9c4e-0c4f1ba30339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:25.820178 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aeb22-ecfb-4f25-9c4e-0c4f1ba30339 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:25.820916 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51aeb22-ecfb-4f25-9c4e-0c4f1ba30339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:25.824297 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51aeb22-ecfb-4f25-9c4e-0c4f1ba30339 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:25.854419 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1385/2798] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:19:25.863203 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938854aa-52be-4240-918c-52e1d3376824 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:25.886430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938854aa-52be-4240-918c-52e1d3376824 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:25.892461 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c770a1-8339-41a4-bcaf-4da9e15fcae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:25.893336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c770a1-8339-41a4-bcaf-4da9e15fcae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:25.893727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c770a1-8339-41a4-bcaf-4da9e15fcae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:25.894145 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c770a1-8339-41a4-bcaf-4da9e15fcae2 admin admin] PATH_INFO: `/v3/projects/c7041a5e4c714f5fa5b77e8b2c3af41b/users/2458d329b68d4a44814293fb42fe0eda/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:25.896579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c770a1-8339-41a4-bcaf-4da9e15fcae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7041a5e4c714f5fa5b77e8b2c3af41b, user_id=2458d329b68d4a44814293fb42fe0eda, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:25.899495 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c770a1-8339-41a4-bcaf-4da9e15fcae2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:25.923771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1414/2799] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:19:25 2022] PUT /identity/v3/projects/c7041a5e4c714f5fa5b77e8b2c3af41b/users/2458d329b68d4a44814293fb42fe0eda/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:19:30.398473 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51aeb22-ecfb-4f25-9c4e-0c4f1ba30339 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:30.403028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d51aeb22-ecfb-4f25-9c4e-0c4f1ba30339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:30.538917 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51aeb22-ecfb-4f25-9c4e-0c4f1ba30339 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:30.625258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ea482-9338-40e3-a207-7ff3b3fccaf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:30.626727 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ea482-9338-40e3-a207-7ff3b3fccaf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:30.627076 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ea482-9338-40e3-a207-7ff3b3fccaf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:30.627617 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ea482-9338-40e3-a207-7ff3b3fccaf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:30.631481 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3c2ea482-9338-40e3-a207-7ff3b3fccaf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:30.749208 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ea482-9338-40e3-a207-7ff3b3fccaf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:30.752985 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ea482-9338-40e3-a207-7ff3b3fccaf2 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:30.766828 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1386/2800] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:19:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:19:31.120875 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c770a1-8339-41a4-bcaf-4da9e15fcae2 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:31.123245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-27c770a1-8339-41a4-bcaf-4da9e15fcae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:31.221947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c770a1-8339-41a4-bcaf-4da9e15fcae2 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:31.232810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b19e1-2e61-4024-b864-12d6a02b36b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:31.233971 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b19e1-2e61-4024-b864-12d6a02b36b7 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:31.233971 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b19e1-2e61-4024-b864-12d6a02b36b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:31.234373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b19e1-2e61-4024-b864-12d6a02b36b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:31.236641 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-dc3b19e1-2e61-4024-b864-12d6a02b36b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:31.338544 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:19:31.404958 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b19e1-2e61-4024-b864-12d6a02b36b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:31.405217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b19e1-2e61-4024-b864-12d6a02b36b7 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:31.430363 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1415/2801] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:19:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:19:32.586947 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d83671-0dbe-413f-82a1-cb87d46ee880 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:32.592857 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d83671-0dbe-413f-82a1-cb87d46ee880 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:32.592857 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d83671-0dbe-413f-82a1-cb87d46ee880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:32.603836 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:19:32.632588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-13d83671-0dbe-413f-82a1-cb87d46ee880 None None] MFA Rules not processed for user `2458d329b68d4a44814293fb42fe0eda`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:19:32.634770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-13d83671-0dbe-413f-82a1-cb87d46ee880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:32.769591 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1387/2802] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:19:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:19:32.777791 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b19e1-2e61-4024-b864-12d6a02b36b7 service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:32.780450 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-dc3b19e1-2e61-4024-b864-12d6a02b36b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:32.801203 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b19e1-2e61-4024-b864-12d6a02b36b7 service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:32.813251 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593d3dd-c326-44f1-8169-3bfbe3c05e5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:32.815195 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593d3dd-c326-44f1-8169-3bfbe3c05e5c service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:32.815866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593d3dd-c326-44f1-8169-3bfbe3c05e5c service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:32.816250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593d3dd-c326-44f1-8169-3bfbe3c05e5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:32.915189 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593d3dd-c326-44f1-8169-3bfbe3c05e5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:32.918085 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593d3dd-c326-44f1-8169-3bfbe3c05e5c service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:32.931127 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1416/2803] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:19:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:19:42.852619 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=31 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:19:42.926430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=48 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:19:51.056769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d83671-0dbe-413f-82a1-cb87d46ee880 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:51.061098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-13d83671-0dbe-413f-82a1-cb87d46ee880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:51.133218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d83671-0dbe-413f-82a1-cb87d46ee880 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:51.139955 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b6bc6-7bb6-4a89-843e-53f1e49d522b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca10dd7ed714edb8b87017deecc41b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:51.140574 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b6bc6-7bb6-4a89-843e-53f1e49d522b service placement] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:51.140892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b6bc6-7bb6-4a89-843e-53f1e49d522b service placement] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:51.141167 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b6bc6-7bb6-4a89-843e-53f1e49d522b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:51.142978 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a50b6bc6-7bb6-4a89-843e-53f1e49d522b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:51.155856 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b6bc6-7bb6-4a89-843e-53f1e49d522b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:51.157453 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b6bc6-7bb6-4a89-843e-53f1e49d522b service placement] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:51.168467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1388/2804] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:19:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:19:58.159144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593d3dd-c326-44f1-8169-3bfbe3c05e5c service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:19:58.168735 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593d3dd-c326-44f1-8169-3bfbe3c05e5c service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:19:58.172587 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b744d6-d18c-42a0-b660-111c463b8b7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:19:58.173265 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b744d6-d18c-42a0-b660-111c463b8b7b service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:19:58.173545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b744d6-d18c-42a0-b660-111c463b8b7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:19:58.173927 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b744d6-d18c-42a0-b660-111c463b8b7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:19:58.175820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-99b744d6-d18c-42a0-b660-111c463b8b7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:19:58.238725 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b744d6-d18c-42a0-b660-111c463b8b7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:19:58.240308 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b744d6-d18c-42a0-b660-111c463b8b7b service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:19:58.250652 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1417/2805] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:19:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:20:01.163703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=32 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:20:02.008405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b6bc6-7bb6-4a89-843e-53f1e49d522b service placement] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:02.010035 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a50b6bc6-7bb6-4a89-843e-53f1e49d522b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:02.025832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b6bc6-7bb6-4a89-843e-53f1e49d522b service placement] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:02.031300 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef2072-eccc-41cb-8001-d804dd63222b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:02.031951 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef2072-eccc-41cb-8001-d804dd63222b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:02.032219 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef2072-eccc-41cb-8001-d804dd63222b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:02.032496 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef2072-eccc-41cb-8001-d804dd63222b admin admin] PATH_INFO: `/v3/users/31f3271adc36406fb303fb169020e740` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:02.033703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef2072-eccc-41cb-8001-d804dd63222b admin admin] RBAC: Authorizing `identity:delete_user(user_id=31f3271adc36406fb303fb169020e740)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:02.035614 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef2072-eccc-41cb-8001-d804dd63222b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:02.210775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-82ef2072-eccc-41cb-8001-d804dd63222b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31f3271adc36406fb303fb169020e740'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:02.224960 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-82ef2072-eccc-41cb-8001-d804dd63222b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31f3271adc36406fb303fb169020e740'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:02.232133 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-82ef2072-eccc-41cb-8001-d804dd63222b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31f3271adc36406fb303fb169020e740'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:02.281446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-82ef2072-eccc-41cb-8001-d804dd63222b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31f3271adc36406fb303fb169020e740'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:02.282101 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-82ef2072-eccc-41cb-8001-d804dd63222b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31f3271adc36406fb303fb169020e740'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:02.288699 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1389/2806] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:20:02 2022] DELETE /identity/v3/users/31f3271adc36406fb303fb169020e740 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:20:02.510935 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b744d6-d18c-42a0-b660-111c463b8b7b service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:02.531487 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b744d6-d18c-42a0-b660-111c463b8b7b service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:02.582865 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a52427-9a41-479f-9c77-f13e9cf40878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:02.583501 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a52427-9a41-479f-9c77-f13e9cf40878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:02.583785 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a52427-9a41-479f-9c77-f13e9cf40878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:02.584106 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a52427-9a41-479f-9c77-f13e9cf40878 admin admin] PATH_INFO: `/v3/projects/5b86ec91335c49aaa672aa85461d2936` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:02.585184 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a52427-9a41-479f-9c77-f13e9cf40878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b86ec91335c49aaa672aa85461d2936)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:02.587105 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a52427-9a41-479f-9c77-f13e9cf40878 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:02.723472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-25a52427-9a41-479f-9c77-f13e9cf40878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b86ec91335c49aaa672aa85461d2936', 'request_id': 'req-25a52427-9a41-479f-9c77-f13e9cf40878'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:02.789660 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-25a52427-9a41-479f-9c77-f13e9cf40878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b86ec91335c49aaa672aa85461d2936', 'request_id': 'req-25a52427-9a41-479f-9c77-f13e9cf40878'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:02.796545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-25a52427-9a41-479f-9c77-f13e9cf40878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b86ec91335c49aaa672aa85461d2936', 'request_id': 'req-25a52427-9a41-479f-9c77-f13e9cf40878'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:02.807231 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-25a52427-9a41-479f-9c77-f13e9cf40878 admin admin] The token cache is being invalidate because project 5b86ec91335c49aaa672aa85461d2936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:20:02.807521 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-25a52427-9a41-479f-9c77-f13e9cf40878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:02.808174 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-25a52427-9a41-479f-9c77-f13e9cf40878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:02.809658 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1418/2807] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:20:02 2022] DELETE /identity/v3/projects/5b86ec91335c49aaa672aa85461d2936 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:20:02.817241 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2d803-a192-411f-9cb9-1039c5cff9f6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:02.817529 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2d803-a192-411f-9cb9-1039c5cff9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:02.817816 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2d803-a192-411f-9cb9-1039c5cff9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:02.851923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-90d2d803-a192-411f-9cb9-1039c5cff9f6 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:20:02.853499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-90d2d803-a192-411f-9cb9-1039c5cff9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:02.919951 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1390/2808] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:20:02.930584 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b698593-7c6f-4a67-8aae-ffd6b5decfad None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:02.931616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b698593-7c6f-4a67-8aae-ffd6b5decfad None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:02.932118 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b698593-7c6f-4a67-8aae-ffd6b5decfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:02.956807 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1b698593-7c6f-4a67-8aae-ffd6b5decfad None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:20:02.958100 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1b698593-7c6f-4a67-8aae-ffd6b5decfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:02.966386 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1419/2809] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:20:02.971006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2d803-a192-411f-9cb9-1039c5cff9f6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:02.981408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2d803-a192-411f-9cb9-1039c5cff9f6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:02.986194 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c54e2-d303-4713-9f9e-4ab4f0e2d07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:02.986830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c54e2-d303-4713-9f9e-4ab4f0e2d07a admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:02.987123 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c54e2-d303-4713-9f9e-4ab4f0e2d07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:02.987403 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c54e2-d303-4713-9f9e-4ab4f0e2d07a admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:02.988084 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c54e2-d303-4713-9f9e-4ab4f0e2d07a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:20:02.988390 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c54e2-d303-4713-9f9e-4ab4f0e2d07a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:02.990051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c54e2-d303-4713-9f9e-4ab4f0e2d07a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:03.003530 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1391/2810] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 12:20:03.011783 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b698593-7c6f-4a67-8aae-ffd6b5decfad None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:03.022258 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b698593-7c6f-4a67-8aae-ffd6b5decfad None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:03.026859 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db795a13-03e9-4228-84a8-feeeeb53ca28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:03.027470 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db795a13-03e9-4228-84a8-feeeeb53ca28 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:03.027750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db795a13-03e9-4228-84a8-feeeeb53ca28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:03.028021 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db795a13-03e9-4228-84a8-feeeeb53ca28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:03.028657 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db795a13-03e9-4228-84a8-feeeeb53ca28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:03.030645 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db795a13-03e9-4228-84a8-feeeeb53ca28 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:03.048972 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:20:03.116849 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1420/2811] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:20:03 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:20:03.122405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c54e2-d303-4713-9f9e-4ab4f0e2d07a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:03.132980 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c54e2-d303-4713-9f9e-4ab4f0e2d07a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:03.193642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c278a-001c-44ee-8d4c-7bbf93855d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:03.194086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c278a-001c-44ee-8d4c-7bbf93855d64 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:03.194086 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c278a-001c-44ee-8d4c-7bbf93855d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:03.194678 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c278a-001c-44ee-8d4c-7bbf93855d64 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:03.195008 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c278a-001c-44ee-8d4c-7bbf93855d64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:03.196941 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c278a-001c-44ee-8d4c-7bbf93855d64 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:03.205113 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:20:03.241396 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1392/2812] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:20:03 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:20:03.247851 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db795a13-03e9-4228-84a8-feeeeb53ca28 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:03.259878 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db795a13-03e9-4228-84a8-feeeeb53ca28 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:03.267300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881b1a3-56a0-49a3-84c5-b3669af8a175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:03.267300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881b1a3-56a0-49a3-84c5-b3669af8a175 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:03.267300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881b1a3-56a0-49a3-84c5-b3669af8a175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:03.267300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881b1a3-56a0-49a3-84c5-b3669af8a175 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:03.267736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881b1a3-56a0-49a3-84c5-b3669af8a175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:03.268026 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881b1a3-56a0-49a3-84c5-b3669af8a175 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:03.278744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1421/2813] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:20:03.285555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c278a-001c-44ee-8d4c-7bbf93855d64 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:03.296574 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c278a-001c-44ee-8d4c-7bbf93855d64 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:03.301363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ff053-e832-458a-93f2-29081f819077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:03.301940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ff053-e832-458a-93f2-29081f819077 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:03.302231 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ff053-e832-458a-93f2-29081f819077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:03.302511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ff053-e832-458a-93f2-29081f819077 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:03.303856 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ff053-e832-458a-93f2-29081f819077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:03.304808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ff053-e832-458a-93f2-29081f819077 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:03.316451 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1393/2814] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:20:03.321598 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881b1a3-56a0-49a3-84c5-b3669af8a175 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:03.333540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881b1a3-56a0-49a3-84c5-b3669af8a175 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:03.339413 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e3763-4897-45be-8dc8-037b6a2e34d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:03.340530 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e3763-4897-45be-8dc8-037b6a2e34d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:03.340869 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e3763-4897-45be-8dc8-037b6a2e34d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:03.341162 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e3763-4897-45be-8dc8-037b6a2e34d8 admin admin] PATH_INFO: `/v3/projects/a9007dc875504694be1e8590a82d6969/users/163fe0c6ced34f058bd5de0b61cea590/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:03.357412 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e3763-4897-45be-8dc8-037b6a2e34d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9007dc875504694be1e8590a82d6969, user_id=163fe0c6ced34f058bd5de0b61cea590, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:03.360820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e3763-4897-45be-8dc8-037b6a2e34d8 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:03.425582 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1422/2815] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:20:03 2022] PUT /identity/v3/projects/a9007dc875504694be1e8590a82d6969/users/163fe0c6ced34f058bd5de0b61cea590/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:20:03.436255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575b951-1800-4e40-8a65-7294af86b3a7 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:03.436613 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575b951-1800-4e40-8a65-7294af86b3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:03.437602 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575b951-1800-4e40-8a65-7294af86b3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:03.470179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-0575b951-1800-4e40-8a65-7294af86b3a7 None None] MFA Rules not processed for user `163fe0c6ced34f058bd5de0b61cea590`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:20:03.474294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0575b951-1800-4e40-8a65-7294af86b3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:03.539572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882a1d4-dad9-41ed-80d9-ed5891da75b8 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:03.540111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882a1d4-dad9-41ed-80d9-ed5891da75b8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:03.540111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882a1d4-dad9-41ed-80d9-ed5891da75b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:03.540477 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1394/2816] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:20:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:20:03.552155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575b951-1800-4e40-8a65-7294af86b3a7 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:03.553458 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0575b951-1800-4e40-8a65-7294af86b3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:03.562102 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-b882a1d4-dad9-41ed-80d9-ed5891da75b8 None None] MFA Rules not processed for user `33f480ef1b8b49df83276a66f445ae85`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:20:03.563319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b882a1d4-dad9-41ed-80d9-ed5891da75b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:03.613717 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575b951-1800-4e40-8a65-7294af86b3a7 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:03.617467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dbec66-1205-4493-b283-617d2d8832db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:03.618118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbec66-1205-4493-b283-617d2d8832db service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:03.618401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbec66-1205-4493-b283-617d2d8832db service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:03.618680 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbec66-1205-4493-b283-617d2d8832db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:03.621336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1423/2817] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:20:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:20:03.632018 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dbec66-1205-4493-b283-617d2d8832db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:03.633652 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dbec66-1205-4493-b283-617d2d8832db service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:03.642900 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1395/2818] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:20:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:20:13.613966 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:20:13.638545 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:INSERT=3,keystone:SELECT=65 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:20:33.835713 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882a1d4-dad9-41ed-80d9-ed5891da75b8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:33.908506 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882a1d4-dad9-41ed-80d9-ed5891da75b8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:33.913134 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9090c-3e8c-46eb-bdb8-aa6439fc232d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:33.913770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9090c-3e8c-46eb-bdb8-aa6439fc232d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:33.914088 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9090c-3e8c-46eb-bdb8-aa6439fc232d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:33.914370 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9090c-3e8c-46eb-bdb8-aa6439fc232d admin admin] PATH_INFO: `/v3/users/163fe0c6ced34f058bd5de0b61cea590` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:33.915508 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9090c-3e8c-46eb-bdb8-aa6439fc232d admin admin] RBAC: Authorizing `identity:delete_user(user_id=163fe0c6ced34f058bd5de0b61cea590)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:33.917432 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9090c-3e8c-46eb-bdb8-aa6439fc232d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:34.089735 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-03e9090c-3e8c-46eb-bdb8-aa6439fc232d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '163fe0c6ced34f058bd5de0b61cea590'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:34.102426 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-03e9090c-3e8c-46eb-bdb8-aa6439fc232d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '163fe0c6ced34f058bd5de0b61cea590'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:34.110036 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-03e9090c-3e8c-46eb-bdb8-aa6439fc232d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '163fe0c6ced34f058bd5de0b61cea590'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:34.183996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-03e9090c-3e8c-46eb-bdb8-aa6439fc232d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '163fe0c6ced34f058bd5de0b61cea590'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:34.184750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-03e9090c-3e8c-46eb-bdb8-aa6439fc232d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '163fe0c6ced34f058bd5de0b61cea590'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:34.191179 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1424/2819] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:20:33 2022] DELETE /identity/v3/users/163fe0c6ced34f058bd5de0b61cea590 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:20:34.195836 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047fa53-02ed-4df5-b0b7-ec071de21e22 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:34.196385 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047fa53-02ed-4df5-b0b7-ec071de21e22 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:34.196385 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047fa53-02ed-4df5-b0b7-ec071de21e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:34.218214 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c047fa53-02ed-4df5-b0b7-ec071de21e22 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:20:34.219456 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c047fa53-02ed-4df5-b0b7-ec071de21e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:34.284459 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1396/2820] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:20:34.296474 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9090c-3e8c-46eb-bdb8-aa6439fc232d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:34.297986 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-03e9090c-3e8c-46eb-bdb8-aa6439fc232d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:34.361711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9090c-3e8c-46eb-bdb8-aa6439fc232d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:34.365977 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece928e-50e4-4c82-9187-d47bee256196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:34.366593 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece928e-50e4-4c82-9187-d47bee256196 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:34.366872 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece928e-50e4-4c82-9187-d47bee256196 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:34.367146 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece928e-50e4-4c82-9187-d47bee256196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:34.376498 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dece928e-50e4-4c82-9187-d47bee256196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:34.377953 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dece928e-50e4-4c82-9187-d47bee256196 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:34.387374 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1425/2821] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:20:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:20:34.756195 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047fa53-02ed-4df5-b0b7-ec071de21e22 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:34.757795 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c047fa53-02ed-4df5-b0b7-ec071de21e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:34.772833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047fa53-02ed-4df5-b0b7-ec071de21e22 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:34.777633 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c8acb-2677-45f8-8d13-e6b4a91c520d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:34.778339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c8acb-2677-45f8-8d13-e6b4a91c520d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:34.778625 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c8acb-2677-45f8-8d13-e6b4a91c520d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:34.778920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c8acb-2677-45f8-8d13-e6b4a91c520d admin admin] PATH_INFO: `/v3/projects/a9007dc875504694be1e8590a82d6969` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:34.781354 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c8acb-2677-45f8-8d13-e6b4a91c520d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9007dc875504694be1e8590a82d6969)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:34.783653 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c8acb-2677-45f8-8d13-e6b4a91c520d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:34.899938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-527c8acb-2677-45f8-8d13-e6b4a91c520d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9007dc875504694be1e8590a82d6969', 'request_id': 'req-527c8acb-2677-45f8-8d13-e6b4a91c520d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:34.954602 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-527c8acb-2677-45f8-8d13-e6b4a91c520d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9007dc875504694be1e8590a82d6969', 'request_id': 'req-527c8acb-2677-45f8-8d13-e6b4a91c520d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:34.960763 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-527c8acb-2677-45f8-8d13-e6b4a91c520d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9007dc875504694be1e8590a82d6969', 'request_id': 'req-527c8acb-2677-45f8-8d13-e6b4a91c520d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:34.972955 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-527c8acb-2677-45f8-8d13-e6b4a91c520d admin admin] The token cache is being invalidate because project a9007dc875504694be1e8590a82d6969 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:20:34.973277 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-527c8acb-2677-45f8-8d13-e6b4a91c520d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:34.974039 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-527c8acb-2677-45f8-8d13-e6b4a91c520d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:34.975661 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1397/2822] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:20:34 2022] DELETE /identity/v3/projects/a9007dc875504694be1e8590a82d6969 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:20:34.980441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8b57d-e258-45e6-b30e-13455bf00f79 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:34.980952 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8b57d-e258-45e6-b30e-13455bf00f79 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:34.981068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8b57d-e258-45e6-b30e-13455bf00f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:35.001523 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a9d8b57d-e258-45e6-b30e-13455bf00f79 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:20:35.002774 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a9d8b57d-e258-45e6-b30e-13455bf00f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:35.020976 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:20:35.063786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1426/2823] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:20:35.073199 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3de05-3853-4b0a-bf00-2e66bad894d6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:35.073482 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3de05-3853-4b0a-bf00-2e66bad894d6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:35.073752 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3de05-3853-4b0a-bf00-2e66bad894d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:35.094271 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-73b3de05-3853-4b0a-bf00-2e66bad894d6 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:20:35.095501 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-73b3de05-3853-4b0a-bf00-2e66bad894d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:35.103429 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1398/2824] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:20:35.107822 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8b57d-e258-45e6-b30e-13455bf00f79 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:35.117331 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8b57d-e258-45e6-b30e-13455bf00f79 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:35.121907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602e4b8-8920-4215-83af-d6ed2ea56d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:35.122513 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602e4b8-8920-4215-83af-d6ed2ea56d50 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:35.122792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602e4b8-8920-4215-83af-d6ed2ea56d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:35.123063 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602e4b8-8920-4215-83af-d6ed2ea56d50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:35.123700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4602e4b8-8920-4215-83af-d6ed2ea56d50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:20:35.124000 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4602e4b8-8920-4215-83af-d6ed2ea56d50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:35.126177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4602e4b8-8920-4215-83af-d6ed2ea56d50 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:35.139012 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1427/2825] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:20:35.143754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3de05-3853-4b0a-bf00-2e66bad894d6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:35.153339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3de05-3853-4b0a-bf00-2e66bad894d6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:35.158372 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336e3d6-c0ff-4745-a56c-a741ecae9653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:35.158767 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336e3d6-c0ff-4745-a56c-a741ecae9653 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:35.159045 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336e3d6-c0ff-4745-a56c-a741ecae9653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:35.159316 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336e3d6-c0ff-4745-a56c-a741ecae9653 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:35.159940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336e3d6-c0ff-4745-a56c-a741ecae9653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:35.161821 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336e3d6-c0ff-4745-a56c-a741ecae9653 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:35.242661 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1399/2826] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:20:35 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:20:35.246432 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602e4b8-8920-4215-83af-d6ed2ea56d50 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:35.258345 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602e4b8-8920-4215-83af-d6ed2ea56d50 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:35.315730 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569ab9e-b2ba-4946-b460-60bd3404f566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:35.316344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569ab9e-b2ba-4946-b460-60bd3404f566 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:35.316626 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569ab9e-b2ba-4946-b460-60bd3404f566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:35.316903 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569ab9e-b2ba-4946-b460-60bd3404f566 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:35.317548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569ab9e-b2ba-4946-b460-60bd3404f566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:35.319616 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569ab9e-b2ba-4946-b460-60bd3404f566 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:35.412492 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1428/2827] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:20:35 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:20:35.416747 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336e3d6-c0ff-4745-a56c-a741ecae9653 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:35.430053 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336e3d6-c0ff-4745-a56c-a741ecae9653 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:35.435910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fda808-095c-4d2a-b346-8b69756ee1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:35.435910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fda808-095c-4d2a-b346-8b69756ee1bd admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:35.436193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fda808-095c-4d2a-b346-8b69756ee1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:35.436408 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fda808-095c-4d2a-b346-8b69756ee1bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:35.437724 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fda808-095c-4d2a-b346-8b69756ee1bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:35.439416 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fda808-095c-4d2a-b346-8b69756ee1bd admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:35.450068 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1400/2828] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:20:35.454254 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569ab9e-b2ba-4946-b460-60bd3404f566 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:35.465309 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569ab9e-b2ba-4946-b460-60bd3404f566 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:35.470031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c5823-e198-402e-8bbb-ab013cb1832f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:35.470638 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c5823-e198-402e-8bbb-ab013cb1832f admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:35.470923 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c5823-e198-402e-8bbb-ab013cb1832f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:35.471224 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c5823-e198-402e-8bbb-ab013cb1832f admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:35.471900 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c5823-e198-402e-8bbb-ab013cb1832f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:35.473614 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c5823-e198-402e-8bbb-ab013cb1832f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:35.485676 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1429/2829] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:20:35.490029 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fda808-095c-4d2a-b346-8b69756ee1bd admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:35.500565 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fda808-095c-4d2a-b346-8b69756ee1bd admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:35.505578 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d8b53-392f-4798-96b0-896b98e27262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:35.506364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d8b53-392f-4798-96b0-896b98e27262 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:35.506646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d8b53-392f-4798-96b0-896b98e27262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:35.506923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d8b53-392f-4798-96b0-896b98e27262 admin admin] PATH_INFO: `/v3/projects/3e2d6052a8984cb99a875953000fb6cf/users/f450887b96424b728e290e9410398114/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:35.523327 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d8b53-392f-4798-96b0-896b98e27262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e2d6052a8984cb99a875953000fb6cf, user_id=f450887b96424b728e290e9410398114, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:35.526078 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d8b53-392f-4798-96b0-896b98e27262 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:35.595488 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1401/2830] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:20:35 2022] PUT /identity/v3/projects/3e2d6052a8984cb99a875953000fb6cf/users/f450887b96424b728e290e9410398114/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:20:35.614151 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe184c2c-b235-4ada-b840-bfb2fa7568e9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:35.614598 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe184c2c-b235-4ada-b840-bfb2fa7568e9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:35.614839 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe184c2c-b235-4ada-b840-bfb2fa7568e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:35.643727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-fe184c2c-b235-4ada-b840-bfb2fa7568e9 None None] MFA Rules not processed for user `f450887b96424b728e290e9410398114`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:20:35.649400 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fe184c2c-b235-4ada-b840-bfb2fa7568e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:35.716567 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1430/2831] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:20:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:20:35.732474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d8b53-392f-4798-96b0-896b98e27262 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:35.734923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b21d8b53-392f-4798-96b0-896b98e27262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:35.756919 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:20:35.800555 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d8b53-392f-4798-96b0-896b98e27262 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:35.804890 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318c144-9ef7-42d2-aa89-7e20cadfdc9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:35.805518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318c144-9ef7-42d2-aa89-7e20cadfdc9b service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:35.805807 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318c144-9ef7-42d2-aa89-7e20cadfdc9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:35.806123 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318c144-9ef7-42d2-aa89-7e20cadfdc9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:35.815980 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318c144-9ef7-42d2-aa89-7e20cadfdc9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:35.817409 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318c144-9ef7-42d2-aa89-7e20cadfdc9b service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:35.827543 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1402/2832] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:20:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:20:36.905130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe184c2c-b235-4ada-b840-bfb2fa7568e9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:36.907337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fe184c2c-b235-4ada-b840-bfb2fa7568e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:36.934795 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:20:36.973084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe184c2c-b235-4ada-b840-bfb2fa7568e9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:36.976754 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50052d18-3f19-4af9-84be-f8b0643211ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:36.977344 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50052d18-3f19-4af9-84be-f8b0643211ab service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:36.977626 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50052d18-3f19-4af9-84be-f8b0643211ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:36.977932 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50052d18-3f19-4af9-84be-f8b0643211ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:36.979686 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-50052d18-3f19-4af9-84be-f8b0643211ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:37.045498 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50052d18-3f19-4af9-84be-f8b0643211ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:37.047513 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50052d18-3f19-4af9-84be-f8b0643211ab service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:37.057393 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1431/2833] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:20:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:20:39.115889 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318c144-9ef7-42d2-aa89-7e20cadfdc9b service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:39.117384 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a318c144-9ef7-42d2-aa89-7e20cadfdc9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:39.195120 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318c144-9ef7-42d2-aa89-7e20cadfdc9b service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:39.201099 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8080e-6910-4801-9b4a-a622a9a49ac2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:39.201724 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8080e-6910-4801-9b4a-a622a9a49ac2 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:39.202176 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8080e-6910-4801-9b4a-a622a9a49ac2 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:39.202491 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8080e-6910-4801-9b4a-a622a9a49ac2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:39.204294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-bda8080e-6910-4801-9b4a-a622a9a49ac2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:39.216897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8080e-6910-4801-9b4a-a622a9a49ac2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:39.218628 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8080e-6910-4801-9b4a-a622a9a49ac2 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:39.229803 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1403/2834] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:20:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:20:39.906587 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50052d18-3f19-4af9-84be-f8b0643211ab service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:39.991546 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50052d18-3f19-4af9-84be-f8b0643211ab service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:39.996196 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04758cc-bc47-49f9-8d3b-580751afb8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:39.996844 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04758cc-bc47-49f9-8d3b-580751afb8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:39.997125 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04758cc-bc47-49f9-8d3b-580751afb8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:39.997399 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04758cc-bc47-49f9-8d3b-580751afb8ed admin admin] PATH_INFO: `/v3/users/f450887b96424b728e290e9410398114` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:39.998606 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04758cc-bc47-49f9-8d3b-580751afb8ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=f450887b96424b728e290e9410398114)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:40.001402 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04758cc-bc47-49f9-8d3b-580751afb8ed admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:40.213367 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a04758cc-bc47-49f9-8d3b-580751afb8ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f450887b96424b728e290e9410398114'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:40.227885 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a04758cc-bc47-49f9-8d3b-580751afb8ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f450887b96424b728e290e9410398114'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:40.232096 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:20:40.235552 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a04758cc-bc47-49f9-8d3b-580751afb8ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f450887b96424b728e290e9410398114'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:40.309162 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a04758cc-bc47-49f9-8d3b-580751afb8ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f450887b96424b728e290e9410398114'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:40.309794 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a04758cc-bc47-49f9-8d3b-580751afb8ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f450887b96424b728e290e9410398114'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:40.316866 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1432/2835] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:20:39 2022] DELETE /identity/v3/users/f450887b96424b728e290e9410398114 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:20:40.321517 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cd5c0-af82-4283-8278-770b9b3aab34 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:40.327960 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cd5c0-af82-4283-8278-770b9b3aab34 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:40.327960 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cd5c0-af82-4283-8278-770b9b3aab34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:40.341006 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4d9cd5c0-af82-4283-8278-770b9b3aab34 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:20:40.342283 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4d9cd5c0-af82-4283-8278-770b9b3aab34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:40.404658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1404/2836] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:20:40.414950 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04758cc-bc47-49f9-8d3b-580751afb8ed admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:40.416326 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a04758cc-bc47-49f9-8d3b-580751afb8ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:40.479565 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04758cc-bc47-49f9-8d3b-580751afb8ed admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:40.483191 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e8134-c268-4a59-bf90-1a398207993e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:40.484451 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e8134-c268-4a59-bf90-1a398207993e service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:40.484730 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e8134-c268-4a59-bf90-1a398207993e service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:40.485002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e8134-c268-4a59-bf90-1a398207993e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:40.494098 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e8134-c268-4a59-bf90-1a398207993e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:40.495536 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e8134-c268-4a59-bf90-1a398207993e service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:40.506882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1433/2837] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:20:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:20:40.999180 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9cd5c0-af82-4283-8278-770b9b3aab34 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:41.000271 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4d9cd5c0-af82-4283-8278-770b9b3aab34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:41.012650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9cd5c0-af82-4283-8278-770b9b3aab34 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:41.017489 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9fad7-d5c1-47b0-a9a4-1f6e1e29d21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:41.018175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9fad7-d5c1-47b0-a9a4-1f6e1e29d21b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:41.018463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9fad7-d5c1-47b0-a9a4-1f6e1e29d21b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:41.018744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9fad7-d5c1-47b0-a9a4-1f6e1e29d21b admin admin] PATH_INFO: `/v3/projects/3e2d6052a8984cb99a875953000fb6cf` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:41.019936 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9fad7-d5c1-47b0-a9a4-1f6e1e29d21b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e2d6052a8984cb99a875953000fb6cf)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:41.022181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9fad7-d5c1-47b0-a9a4-1f6e1e29d21b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:41.164323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:20:41.168175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d8c9fad7-d5c1-47b0-a9a4-1f6e1e29d21b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e2d6052a8984cb99a875953000fb6cf', 'request_id': 'req-d8c9fad7-d5c1-47b0-a9a4-1f6e1e29d21b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:41.275162 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d8c9fad7-d5c1-47b0-a9a4-1f6e1e29d21b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e2d6052a8984cb99a875953000fb6cf', 'request_id': 'req-d8c9fad7-d5c1-47b0-a9a4-1f6e1e29d21b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:41.281763 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d8c9fad7-d5c1-47b0-a9a4-1f6e1e29d21b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e2d6052a8984cb99a875953000fb6cf', 'request_id': 'req-d8c9fad7-d5c1-47b0-a9a4-1f6e1e29d21b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:41.293156 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d8c9fad7-d5c1-47b0-a9a4-1f6e1e29d21b admin admin] The token cache is being invalidate because project 3e2d6052a8984cb99a875953000fb6cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:20:41.293449 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d8c9fad7-d5c1-47b0-a9a4-1f6e1e29d21b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:41.294139 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d8c9fad7-d5c1-47b0-a9a4-1f6e1e29d21b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:20:41.295646 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1405/2838] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:20:40 2022] DELETE /identity/v3/projects/3e2d6052a8984cb99a875953000fb6cf => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:20:41.301214 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eecc7b-c12a-43db-b7e8-6d0528f26df8 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:41.301777 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eecc7b-c12a-43db-b7e8-6d0528f26df8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:41.301777 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eecc7b-c12a-43db-b7e8-6d0528f26df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:41.321636 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a3eecc7b-c12a-43db-b7e8-6d0528f26df8 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:20:41.325420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a3eecc7b-c12a-43db-b7e8-6d0528f26df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:41.385308 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1434/2839] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:20:41.393700 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b01949-60f5-414a-930a-a66c01190543 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:41.394290 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b01949-60f5-414a-930a-a66c01190543 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:41.394290 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b01949-60f5-414a-930a-a66c01190543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:41.416672 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d3b01949-60f5-414a-930a-a66c01190543 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:20:41.417951 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d3b01949-60f5-414a-930a-a66c01190543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:41.427115 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1406/2840] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:20:41.431631 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eecc7b-c12a-43db-b7e8-6d0528f26df8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:41.442201 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eecc7b-c12a-43db-b7e8-6d0528f26df8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:41.446862 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c320106-ffa0-4e6b-815f-5b605546afa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:41.447475 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c320106-ffa0-4e6b-815f-5b605546afa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:41.447750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c320106-ffa0-4e6b-815f-5b605546afa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:41.448042 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c320106-ffa0-4e6b-815f-5b605546afa9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:41.448675 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c320106-ffa0-4e6b-815f-5b605546afa9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:20:41.448983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c320106-ffa0-4e6b-815f-5b605546afa9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:41.450603 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c320106-ffa0-4e6b-815f-5b605546afa9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:41.463769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1435/2841] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:20:41.468765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b01949-60f5-414a-930a-a66c01190543 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:41.478447 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b01949-60f5-414a-930a-a66c01190543 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:41.483570 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e5d08-6f86-479d-ae17-82ab2d710134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:41.484191 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e5d08-6f86-479d-ae17-82ab2d710134 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:41.484479 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e5d08-6f86-479d-ae17-82ab2d710134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:41.484946 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e5d08-6f86-479d-ae17-82ab2d710134 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:41.486050 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e5d08-6f86-479d-ae17-82ab2d710134 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:41.489422 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e5d08-6f86-479d-ae17-82ab2d710134 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:41.593933 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1407/2842] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:20:41 2022] POST /identity/v3/projects => generated 365 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:20:41.598966 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c320106-ffa0-4e6b-815f-5b605546afa9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:41.610191 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c320106-ffa0-4e6b-815f-5b605546afa9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:41.678907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f01c83-71eb-4125-a9de-cc1b70c232f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:41.679711 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f01c83-71eb-4125-a9de-cc1b70c232f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:41.679975 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f01c83-71eb-4125-a9de-cc1b70c232f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:41.680259 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f01c83-71eb-4125-a9de-cc1b70c232f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:41.681066 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f01c83-71eb-4125-a9de-cc1b70c232f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:41.684373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f01c83-71eb-4125-a9de-cc1b70c232f6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:41.767945 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1436/2843] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:20:41 2022] POST /identity/v3/users => generated 283 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:20:41.774226 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e5d08-6f86-479d-ae17-82ab2d710134 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:41.785405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e5d08-6f86-479d-ae17-82ab2d710134 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:41.790222 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb3b6a-6625-4d30-8ed6-5601b063edb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:41.790830 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb3b6a-6625-4d30-8ed6-5601b063edb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:41.791138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb3b6a-6625-4d30-8ed6-5601b063edb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:41.791424 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb3b6a-6625-4d30-8ed6-5601b063edb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:41.792058 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb3b6a-6625-4d30-8ed6-5601b063edb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:41.793758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb3b6a-6625-4d30-8ed6-5601b063edb3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:41.805084 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1408/2844] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:20:41.809242 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f01c83-71eb-4125-a9de-cc1b70c232f6 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:41.822063 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f01c83-71eb-4125-a9de-cc1b70c232f6 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:41.828733 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b036b-aea3-466e-9379-806f3c7cba7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:41.829448 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b036b-aea3-466e-9379-806f3c7cba7c admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:41.829966 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b036b-aea3-466e-9379-806f3c7cba7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:41.830453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b036b-aea3-466e-9379-806f3c7cba7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:41.831296 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b036b-aea3-466e-9379-806f3c7cba7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:41.833024 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b036b-aea3-466e-9379-806f3c7cba7c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:41.834765 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:20:41.848752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1437/2845] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:20:41.851873 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb3b6a-6625-4d30-8ed6-5601b063edb3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:41.862618 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb3b6a-6625-4d30-8ed6-5601b063edb3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:41.867465 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdb06a-ecca-4531-b0ee-4dcb175429c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:41.868137 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdb06a-ecca-4531-b0ee-4dcb175429c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:41.868418 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdb06a-ecca-4531-b0ee-4dcb175429c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:41.868694 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdb06a-ecca-4531-b0ee-4dcb175429c8 admin admin] PATH_INFO: `/v3/projects/4d19d13ed81d4e0bb97485b1a5d60fa8/users/62c586e574d84f2ea42a5ac995be9f2f/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:41.891568 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdb06a-ecca-4531-b0ee-4dcb175429c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d19d13ed81d4e0bb97485b1a5d60fa8, user_id=62c586e574d84f2ea42a5ac995be9f2f, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:41.894318 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdb06a-ecca-4531-b0ee-4dcb175429c8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:41.972205 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1409/2846] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:20:41 2022] PUT /identity/v3/projects/4d19d13ed81d4e0bb97485b1a5d60fa8/users/62c586e574d84f2ea42a5ac995be9f2f/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:20:41.983671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ee02c-fce5-41e1-ae56-da16d2b4040b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:41.984117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ee02c-fce5-41e1-ae56-da16d2b4040b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:41.984393 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ee02c-fce5-41e1-ae56-da16d2b4040b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:42.015744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-cd9ee02c-fce5-41e1-ae56-da16d2b4040b None None] MFA Rules not processed for user `62c586e574d84f2ea42a5ac995be9f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:20:42.132232 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cd9ee02c-fce5-41e1-ae56-da16d2b4040b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:42.132232 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1438/2847] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:20:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:20:42.132437 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdb06a-ecca-4531-b0ee-4dcb175429c8 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:20:42.132437 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-52fdb06a-ecca-4531-b0ee-4dcb175429c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:20:42.174169 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdb06a-ecca-4531-b0ee-4dcb175429c8 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:20:42.179641 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ded1d-8b1f-4b04-bf93-53f035b3bd1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:20:42.180285 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ded1d-8b1f-4b04-bf93-53f035b3bd1e service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:20:42.180571 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ded1d-8b1f-4b04-bf93-53f035b3bd1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:20:42.180858 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ded1d-8b1f-4b04-bf93-53f035b3bd1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:20:42.190771 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ded1d-8b1f-4b04-bf93-53f035b3bd1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:20:42.192263 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ded1d-8b1f-4b04-bf93-53f035b3bd1e service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:20:42.201783 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1410/2848] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:20:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:20:52.096048 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=34 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:20:52.197365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=61,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:21:39.679301 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ee02c-fce5-41e1-ae56-da16d2b4040b None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:21:39.747185 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ee02c-fce5-41e1-ae56-da16d2b4040b None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:21:39.751721 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371bde1-cfd5-49e5-b19b-f8d09e7fd5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:21:39.752379 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371bde1-cfd5-49e5-b19b-f8d09e7fd5f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:21:39.752681 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371bde1-cfd5-49e5-b19b-f8d09e7fd5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:21:39.752960 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371bde1-cfd5-49e5-b19b-f8d09e7fd5f6 admin admin] PATH_INFO: `/v3/users/62c586e574d84f2ea42a5ac995be9f2f` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:21:39.754140 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371bde1-cfd5-49e5-b19b-f8d09e7fd5f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62c586e574d84f2ea42a5ac995be9f2f)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:21:39.756049 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371bde1-cfd5-49e5-b19b-f8d09e7fd5f6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:21:39.993470 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7371bde1-cfd5-49e5-b19b-f8d09e7fd5f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62c586e574d84f2ea42a5ac995be9f2f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:21:40.006550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7371bde1-cfd5-49e5-b19b-f8d09e7fd5f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62c586e574d84f2ea42a5ac995be9f2f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:21:40.013701 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7371bde1-cfd5-49e5-b19b-f8d09e7fd5f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62c586e574d84f2ea42a5ac995be9f2f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:21:40.091329 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7371bde1-cfd5-49e5-b19b-f8d09e7fd5f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62c586e574d84f2ea42a5ac995be9f2f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:21:40.091991 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7371bde1-cfd5-49e5-b19b-f8d09e7fd5f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62c586e574d84f2ea42a5ac995be9f2f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:21:40.098696 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1439/2849] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:21:39 2022] DELETE /identity/v3/users/62c586e574d84f2ea42a5ac995be9f2f => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:21:40.111308 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52442df-ad98-4062-8f9c-c30345b43d79 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:21:40.112101 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52442df-ad98-4062-8f9c-c30345b43d79 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:21:40.112101 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52442df-ad98-4062-8f9c-c30345b43d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:21:40.135112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c52442df-ad98-4062-8f9c-c30345b43d79 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:21:40.136355 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c52442df-ad98-4062-8f9c-c30345b43d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:21:40.202914 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1411/2850] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:21:40.213293 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371bde1-cfd5-49e5-b19b-f8d09e7fd5f6 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:21:40.214830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7371bde1-cfd5-49e5-b19b-f8d09e7fd5f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:21:40.277556 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371bde1-cfd5-49e5-b19b-f8d09e7fd5f6 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:21:40.281087 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130912a-303f-404e-9662-8c907c765c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:21:40.281695 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130912a-303f-404e-9662-8c907c765c55 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:21:40.282011 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130912a-303f-404e-9662-8c907c765c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:21:40.282810 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130912a-303f-404e-9662-8c907c765c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:21:40.292488 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130912a-303f-404e-9662-8c907c765c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:21:40.293947 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130912a-303f-404e-9662-8c907c765c55 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:21:40.304589 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1440/2851] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:21:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:21:40.732825 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52442df-ad98-4062-8f9c-c30345b43d79 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:21:40.734747 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c52442df-ad98-4062-8f9c-c30345b43d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:21:40.746708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52442df-ad98-4062-8f9c-c30345b43d79 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:21:40.751798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbacc67-115f-42ea-944c-2f90bf4970ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:21:40.752471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbacc67-115f-42ea-944c-2f90bf4970ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:21:40.752814 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbacc67-115f-42ea-944c-2f90bf4970ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:21:40.753233 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbacc67-115f-42ea-944c-2f90bf4970ac admin admin] PATH_INFO: `/v3/projects/4d19d13ed81d4e0bb97485b1a5d60fa8` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:21:40.754321 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbacc67-115f-42ea-944c-2f90bf4970ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d19d13ed81d4e0bb97485b1a5d60fa8)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:21:40.756308 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbacc67-115f-42ea-944c-2f90bf4970ac admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:21:40.893776 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1dbacc67-115f-42ea-944c-2f90bf4970ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d19d13ed81d4e0bb97485b1a5d60fa8', 'request_id': 'req-1dbacc67-115f-42ea-944c-2f90bf4970ac'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:21:41.011765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1dbacc67-115f-42ea-944c-2f90bf4970ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d19d13ed81d4e0bb97485b1a5d60fa8', 'request_id': 'req-1dbacc67-115f-42ea-944c-2f90bf4970ac'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:21:41.018754 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1dbacc67-115f-42ea-944c-2f90bf4970ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d19d13ed81d4e0bb97485b1a5d60fa8', 'request_id': 'req-1dbacc67-115f-42ea-944c-2f90bf4970ac'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:21:41.030440 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1dbacc67-115f-42ea-944c-2f90bf4970ac admin admin] The token cache is being invalidate because project 4d19d13ed81d4e0bb97485b1a5d60fa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:21:41.030739 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1dbacc67-115f-42ea-944c-2f90bf4970ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:21:41.031395 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1dbacc67-115f-42ea-944c-2f90bf4970ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:21:41.032887 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1412/2852] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:21:40 2022] DELETE /identity/v3/projects/4d19d13ed81d4e0bb97485b1a5d60fa8 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:21:41.037936 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a205fd4-2773-4d43-a498-e56e866560b9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:21:41.038512 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a205fd4-2773-4d43-a498-e56e866560b9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:21:41.038512 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a205fd4-2773-4d43-a498-e56e866560b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:21:41.057634 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3a205fd4-2773-4d43-a498-e56e866560b9 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:21:41.058869 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3a205fd4-2773-4d43-a498-e56e866560b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:21:41.075830 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:21:41.118873 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1441/2853] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:21:41.127528 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ef797-d5a2-4477-a352-d28a25cb7c29 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:21:41.128047 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ef797-d5a2-4477-a352-d28a25cb7c29 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:21:41.128155 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ef797-d5a2-4477-a352-d28a25cb7c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:21:41.149584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-7c5ef797-d5a2-4477-a352-d28a25cb7c29 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:21:41.150899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7c5ef797-d5a2-4477-a352-d28a25cb7c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:21:41.159263 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1413/2854] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:21:41.165219 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a205fd4-2773-4d43-a498-e56e866560b9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:21:41.175608 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a205fd4-2773-4d43-a498-e56e866560b9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:21:41.180528 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6502a-3b87-47af-b505-cb221c2f1b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:21:41.181243 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6502a-3b87-47af-b505-cb221c2f1b3a admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:21:41.181526 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6502a-3b87-47af-b505-cb221c2f1b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:21:41.181803 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6502a-3b87-47af-b505-cb221c2f1b3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:21:41.182489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6502a-3b87-47af-b505-cb221c2f1b3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:21:41.182792 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6502a-3b87-47af-b505-cb221c2f1b3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:21:41.184455 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6502a-3b87-47af-b505-cb221c2f1b3a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:21:41.197256 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1442/2855] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:21:41.201897 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ef797-d5a2-4477-a352-d28a25cb7c29 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:21:41.212159 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ef797-d5a2-4477-a352-d28a25cb7c29 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:21:41.217400 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6135a6-df9f-47d1-9820-8e373dffeeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:21:41.218098 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6135a6-df9f-47d1-9820-8e373dffeeda admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:21:41.218384 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6135a6-df9f-47d1-9820-8e373dffeeda admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:21:41.218662 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6135a6-df9f-47d1-9820-8e373dffeeda admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:21:41.219318 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6135a6-df9f-47d1-9820-8e373dffeeda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:21:41.221538 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6135a6-df9f-47d1-9820-8e373dffeeda admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:21:41.305596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1414/2856] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:21:41 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 12:21:41.309832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6502a-3b87-47af-b505-cb221c2f1b3a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:21:41.321269 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6502a-3b87-47af-b505-cb221c2f1b3a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:21:41.380300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529436f-34d4-4d2c-a364-df7ccd642e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:21:41.380961 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529436f-34d4-4d2c-a364-df7ccd642e09 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:21:41.381250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529436f-34d4-4d2c-a364-df7ccd642e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:21:41.381544 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529436f-34d4-4d2c-a364-df7ccd642e09 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:21:41.382245 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529436f-34d4-4d2c-a364-df7ccd642e09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:21:41.384170 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529436f-34d4-4d2c-a364-df7ccd642e09 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:21:41.492417 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1443/2857] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:21:41 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:21:41.496990 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6135a6-df9f-47d1-9820-8e373dffeeda admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:21:41.508982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6135a6-df9f-47d1-9820-8e373dffeeda admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:21:41.513786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934b662-0b14-43fb-ba4b-fea19e3eb538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:21:41.514376 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934b662-0b14-43fb-ba4b-fea19e3eb538 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:21:41.514660 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934b662-0b14-43fb-ba4b-fea19e3eb538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:21:41.514940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934b662-0b14-43fb-ba4b-fea19e3eb538 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:21:41.515588 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934b662-0b14-43fb-ba4b-fea19e3eb538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:21:41.517259 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934b662-0b14-43fb-ba4b-fea19e3eb538 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:21:41.528577 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1415/2858] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:21:41.532768 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529436f-34d4-4d2c-a364-df7ccd642e09 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:21:41.543426 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529436f-34d4-4d2c-a364-df7ccd642e09 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:21:41.548805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148dfbb-b53a-4dd8-8016-237ab30e96d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:21:41.549411 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148dfbb-b53a-4dd8-8016-237ab30e96d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:21:41.549700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148dfbb-b53a-4dd8-8016-237ab30e96d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:21:41.550022 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148dfbb-b53a-4dd8-8016-237ab30e96d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:21:41.550697 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148dfbb-b53a-4dd8-8016-237ab30e96d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:21:41.552294 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148dfbb-b53a-4dd8-8016-237ab30e96d1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:21:41.564506 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1444/2859] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:21:41.569126 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934b662-0b14-43fb-ba4b-fea19e3eb538 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:21:41.579415 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934b662-0b14-43fb-ba4b-fea19e3eb538 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:21:41.584217 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657619c0-7848-4410-a6bf-ef884aafac30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:21:41.584872 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657619c0-7848-4410-a6bf-ef884aafac30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:21:41.585159 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657619c0-7848-4410-a6bf-ef884aafac30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:21:41.585441 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657619c0-7848-4410-a6bf-ef884aafac30 admin admin] PATH_INFO: `/v3/projects/33a46571f6a64b65aa0e37f67cfdae0c/users/0bbebfb828cb410a818a87a9bb5717cd/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:21:41.601113 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657619c0-7848-4410-a6bf-ef884aafac30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33a46571f6a64b65aa0e37f67cfdae0c, user_id=0bbebfb828cb410a818a87a9bb5717cd, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:21:41.604105 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657619c0-7848-4410-a6bf-ef884aafac30 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:21:41.680470 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1416/2860] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:21:41 2022] PUT /identity/v3/projects/33a46571f6a64b65aa0e37f67cfdae0c/users/0bbebfb828cb410a818a87a9bb5717cd/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:21:41.700191 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ce734-6240-478d-8ec2-dcd5513c0f91 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:21:41.700478 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ce734-6240-478d-8ec2-dcd5513c0f91 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:21:41.700753 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ce734-6240-478d-8ec2-dcd5513c0f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:21:41.730579 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-478ce734-6240-478d-8ec2-dcd5513c0f91 None None] MFA Rules not processed for user `0bbebfb828cb410a818a87a9bb5717cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:21:41.731699 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-478ce734-6240-478d-8ec2-dcd5513c0f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:21:41.797099 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1445/2861] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:21:41.809104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657619c0-7848-4410-a6bf-ef884aafac30 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:21:41.810833 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-657619c0-7848-4410-a6bf-ef884aafac30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:21:41.833770 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:21:41.872119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657619c0-7848-4410-a6bf-ef884aafac30 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:21:41.876021 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8887cd-c351-4fb4-8c49-267940f12821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:21:41.876627 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8887cd-c351-4fb4-8c49-267940f12821 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:21:41.876913 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8887cd-c351-4fb4-8c49-267940f12821 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:21:41.877191 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8887cd-c351-4fb4-8c49-267940f12821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:21:41.885701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8887cd-c351-4fb4-8c49-267940f12821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:21:41.887171 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8887cd-c351-4fb4-8c49-267940f12821 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:21:41.897895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1417/2862] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:21:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:21:44.758519 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b890693-8533-425b-8ff4-242be6d6b85b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:21:44.759481 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b890693-8533-425b-8ff4-242be6d6b85b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:21:44.759481 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b890693-8533-425b-8ff4-242be6d6b85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:21:44.782249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9b890693-8533-425b-8ff4-242be6d6b85b None None] MFA Rules not processed for user `0bbebfb828cb410a818a87a9bb5717cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:21:44.783541 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9b890693-8533-425b-8ff4-242be6d6b85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:21:44.791032 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1446/2863] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:21:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:21:44.831698 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8887cd-c351-4fb4-8c49-267940f12821 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:21:44.833994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7b8887cd-c351-4fb4-8c49-267940f12821 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:21:44.917432 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8887cd-c351-4fb4-8c49-267940f12821 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:21:44.921199 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536b1c7-1d8b-43ac-9f38-13ac086fe3ce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b05d0e65e9645f3810a6d96c8fed9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:21:44.921811 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536b1c7-1d8b-43ac-9f38-13ac086fe3ce service glance] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:21:44.922121 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536b1c7-1d8b-43ac-9f38-13ac086fe3ce service glance] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:21:44.922405 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536b1c7-1d8b-43ac-9f38-13ac086fe3ce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:21:44.924190 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d536b1c7-1d8b-43ac-9f38-13ac086fe3ce service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:21:45.000343 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536b1c7-1d8b-43ac-9f38-13ac086fe3ce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:21:45.001645 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536b1c7-1d8b-43ac-9f38-13ac086fe3ce service glance] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:21:45.013327 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1418/2864] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:21:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:21:45.507201 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b890693-8533-425b-8ff4-242be6d6b85b None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:21:45.508646 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9b890693-8533-425b-8ff4-242be6d6b85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:21:45.522912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:21:45.525653 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536b1c7-1d8b-43ac-9f38-13ac086fe3ce service glance] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:21:45.527141 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d536b1c7-1d8b-43ac-9f38-13ac086fe3ce service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:21:45.581204 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:21:45.582631 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b890693-8533-425b-8ff4-242be6d6b85b None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:21:45.588162 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b23f19-09b7-40f4-8884-0eaf71d5d510 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca10dd7ed714edb8b87017deecc41b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:21:45.588781 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b23f19-09b7-40f4-8884-0eaf71d5d510 service placement] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:21:45.589062 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b23f19-09b7-40f4-8884-0eaf71d5d510 service placement] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:21:45.589337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b23f19-09b7-40f4-8884-0eaf71d5d510 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:21:45.591083 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c8b23f19-09b7-40f4-8884-0eaf71d5d510 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:21:45.591801 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536b1c7-1d8b-43ac-9f38-13ac086fe3ce service glance] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:21:45.596081 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef82bfd-7ff1-46d6-ae34-6185069f2013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:21:45.596693 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef82bfd-7ff1-46d6-ae34-6185069f2013 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:21:45.596983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef82bfd-7ff1-46d6-ae34-6185069f2013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:21:45.597264 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef82bfd-7ff1-46d6-ae34-6185069f2013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:21:45.599554 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fef82bfd-7ff1-46d6-ae34-6185069f2013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:21:45.603956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b23f19-09b7-40f4-8884-0eaf71d5d510 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:21:45.605853 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b23f19-09b7-40f4-8884-0eaf71d5d510 service placement] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:21:45.614854 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef82bfd-7ff1-46d6-ae34-6185069f2013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:21:45.616303 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef82bfd-7ff1-46d6-ae34-6185069f2013 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:21:45.617467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1447/2865] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:21:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:21:45.628121 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1419/2866] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:21:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:21:55.612018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=27 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:21:55.633998 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=6 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:22:06.359233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab28524-a112-42a3-9e29-f70e726d1652 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:06.359769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab28524-a112-42a3-9e29-f70e726d1652 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:06.359769 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab28524-a112-42a3-9e29-f70e726d1652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:06.379702 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2ab28524-a112-42a3-9e29-f70e726d1652 None None] MFA Rules not processed for user `d76a164ed7e14d9eb2db016b68a84d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:22:06.380916 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2ab28524-a112-42a3-9e29-f70e726d1652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:06.439261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1448/2867] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 14 12:22:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:22:06.597898 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef82bfd-7ff1-46d6-ae34-6185069f2013 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:06.599431 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fef82bfd-7ff1-46d6-ae34-6185069f2013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:06.667826 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef82bfd-7ff1-46d6-ae34-6185069f2013 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:06.673219 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396470da-fa79-431b-a1aa-2cb104f8341a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1047c7d18c1466d989cd895117cd807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:06.673829 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396470da-fa79-431b-a1aa-2cb104f8341a service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:06.674146 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396470da-fa79-431b-a1aa-2cb104f8341a service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:06.674427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396470da-fa79-431b-a1aa-2cb104f8341a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:06.683314 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396470da-fa79-431b-a1aa-2cb104f8341a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:06.684910 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396470da-fa79-431b-a1aa-2cb104f8341a service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:06.693020 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1420/2868] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 14 12:22:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 12:22:07.469784 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bacee2-31a1-4a9f-b63b-c60be0f9e407 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:07.470332 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bacee2-31a1-4a9f-b63b-c60be0f9e407 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:07.470481 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bacee2-31a1-4a9f-b63b-c60be0f9e407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:07.491002 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e4bacee2-31a1-4a9f-b63b-c60be0f9e407 None None] MFA Rules not processed for user `d76a164ed7e14d9eb2db016b68a84d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:22:07.492298 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e4bacee2-31a1-4a9f-b63b-c60be0f9e407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:07.499778 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1449/2869] 10.222.0.62 () {58 vars in 967 bytes} [Wed Sep 14 12:22:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:22:07.512128 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396470da-fa79-431b-a1aa-2cb104f8341a service swift] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:07.523425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396470da-fa79-431b-a1aa-2cb104f8341a service swift] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:07.528973 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7854fe-ae3f-46a9-8a07-a529be617a4e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1047c7d18c1466d989cd895117cd807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:07.529569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7854fe-ae3f-46a9-8a07-a529be617a4e service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:07.529881 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7854fe-ae3f-46a9-8a07-a529be617a4e service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:07.530161 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7854fe-ae3f-46a9-8a07-a529be617a4e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:07.538371 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7854fe-ae3f-46a9-8a07-a529be617a4e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:07.539950 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7854fe-ae3f-46a9-8a07-a529be617a4e service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:07.549401 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1421/2870] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 14 12:22:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 12:22:08.200340 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b7802-3952-426f-84ec-84be4b33da7a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:08.200628 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b7802-3952-426f-84ec-84be4b33da7a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:08.200907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b7802-3952-426f-84ec-84be4b33da7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:08.222589 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-651b7802-3952-426f-84ec-84be4b33da7a None None] MFA Rules not processed for user `d76a164ed7e14d9eb2db016b68a84d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:22:08.223808 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-651b7802-3952-426f-84ec-84be4b33da7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:08.230980 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1450/2871] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 14 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:22:08.242949 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7854fe-ae3f-46a9-8a07-a529be617a4e service swift] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:08.254284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7854fe-ae3f-46a9-8a07-a529be617a4e service swift] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:08.259533 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217c4fb-ae74-4a14-a676-d79632cf6da9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1047c7d18c1466d989cd895117cd807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:08.260165 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217c4fb-ae74-4a14-a676-d79632cf6da9 service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:08.261001 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217c4fb-ae74-4a14-a676-d79632cf6da9 service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:08.261326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217c4fb-ae74-4a14-a676-d79632cf6da9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:08.269752 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6217c4fb-ae74-4a14-a676-d79632cf6da9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:08.271356 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6217c4fb-ae74-4a14-a676-d79632cf6da9 service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:08.279392 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1422/2872] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 14 12:22:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 12:22:08.872890 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836360d-68c7-4a8a-a618-fbac3d162b8e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:08.873398 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836360d-68c7-4a8a-a618-fbac3d162b8e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:08.873511 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836360d-68c7-4a8a-a618-fbac3d162b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:08.893181 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9836360d-68c7-4a8a-a618-fbac3d162b8e None None] MFA Rules not processed for user `d76a164ed7e14d9eb2db016b68a84d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:22:08.894813 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9836360d-68c7-4a8a-a618-fbac3d162b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:08.902891 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1451/2873] 10.222.0.62 () {58 vars in 967 bytes} [Wed Sep 14 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:22:08.915797 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217c4fb-ae74-4a14-a676-d79632cf6da9 service swift] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:08.927081 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217c4fb-ae74-4a14-a676-d79632cf6da9 service swift] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:08.932564 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc57b0-498e-4859-9e19-fcec9910d3fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1047c7d18c1466d989cd895117cd807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:08.933784 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc57b0-498e-4859-9e19-fcec9910d3fe service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:08.934144 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc57b0-498e-4859-9e19-fcec9910d3fe service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:08.934427 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc57b0-498e-4859-9e19-fcec9910d3fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:08.943747 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc57b0-498e-4859-9e19-fcec9910d3fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:08.949284 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc57b0-498e-4859-9e19-fcec9910d3fe service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:08.954963 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1423/2874] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 14 12:22:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 12:22:10.296752 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836360d-68c7-4a8a-a618-fbac3d162b8e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:10.307222 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836360d-68c7-4a8a-a618-fbac3d162b8e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:10.311408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbd207-e6ee-44d6-a082-6c68580e3ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:10.312015 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbd207-e6ee-44d6-a082-6c68580e3ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:10.312370 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbd207-e6ee-44d6-a082-6c68580e3ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:10.312578 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbd207-e6ee-44d6-a082-6c68580e3ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:10.314404 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7fcbd207-e6ee-44d6-a082-6c68580e3ae4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:10.325188 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcbd207-e6ee-44d6-a082-6c68580e3ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:10.326920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcbd207-e6ee-44d6-a082-6c68580e3ae4 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:10.336908 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1452/2875] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:22:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:22:11.697478 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc57b0-498e-4859-9e19-fcec9910d3fe service swift] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:11.759320 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc57b0-498e-4859-9e19-fcec9910d3fe service swift] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:11.764885 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6b037-497c-4aab-884c-e65cf496e69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:11.765510 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6b037-497c-4aab-884c-e65cf496e69b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:11.765788 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6b037-497c-4aab-884c-e65cf496e69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:11.766090 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6b037-497c-4aab-884c-e65cf496e69b admin admin] PATH_INFO: `/v3/users/0bbebfb828cb410a818a87a9bb5717cd` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:11.767195 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6b037-497c-4aab-884c-e65cf496e69b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bbebfb828cb410a818a87a9bb5717cd)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:11.769100 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6b037-497c-4aab-884c-e65cf496e69b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:11.951322 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-34c6b037-497c-4aab-884c-e65cf496e69b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bbebfb828cb410a818a87a9bb5717cd'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:11.955955 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:22:11.964018 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-34c6b037-497c-4aab-884c-e65cf496e69b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bbebfb828cb410a818a87a9bb5717cd'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:11.970920 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-34c6b037-497c-4aab-884c-e65cf496e69b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bbebfb828cb410a818a87a9bb5717cd'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:12.056892 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-34c6b037-497c-4aab-884c-e65cf496e69b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bbebfb828cb410a818a87a9bb5717cd'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:12.057509 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-34c6b037-497c-4aab-884c-e65cf496e69b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bbebfb828cb410a818a87a9bb5717cd'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:12.064326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1424/2876] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:22:11 2022] DELETE /identity/v3/users/0bbebfb828cb410a818a87a9bb5717cd => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:22:12.069043 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ecd1f-8ae9-428a-a79d-c10f2459562f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:12.069562 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ecd1f-8ae9-428a-a79d-c10f2459562f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:12.069668 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ecd1f-8ae9-428a-a79d-c10f2459562f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:12.088727 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-055ecd1f-8ae9-428a-a79d-c10f2459562f None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:22:12.090049 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-055ecd1f-8ae9-428a-a79d-c10f2459562f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:12.148111 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1453/2877] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:22:12.159193 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6b037-497c-4aab-884c-e65cf496e69b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:12.159570 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-34c6b037-497c-4aab-884c-e65cf496e69b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:12.219422 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6b037-497c-4aab-884c-e65cf496e69b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:12.223156 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d085b6-450a-4f63-b9cf-cf8ea6d0e978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:12.223787 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d085b6-450a-4f63-b9cf-cf8ea6d0e978 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:12.224092 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d085b6-450a-4f63-b9cf-cf8ea6d0e978 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:12.224365 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d085b6-450a-4f63-b9cf-cf8ea6d0e978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:12.234002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d085b6-450a-4f63-b9cf-cf8ea6d0e978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:12.235433 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d085b6-450a-4f63-b9cf-cf8ea6d0e978 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:12.246302 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1425/2878] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:22:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:22:12.475319 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ecd1f-8ae9-428a-a79d-c10f2459562f None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:12.476778 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-055ecd1f-8ae9-428a-a79d-c10f2459562f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:12.489902 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ecd1f-8ae9-428a-a79d-c10f2459562f None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:12.494590 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f677d0-a3ae-45ce-8b87-3c028857614b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:12.495937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f677d0-a3ae-45ce-8b87-3c028857614b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:12.496277 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f677d0-a3ae-45ce-8b87-3c028857614b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:12.496739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f677d0-a3ae-45ce-8b87-3c028857614b admin admin] PATH_INFO: `/v3/projects/33a46571f6a64b65aa0e37f67cfdae0c` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:12.498236 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f677d0-a3ae-45ce-8b87-3c028857614b admin admin] RBAC: Authorizing `identity:delete_project(project_id=33a46571f6a64b65aa0e37f67cfdae0c)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:12.500374 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f677d0-a3ae-45ce-8b87-3c028857614b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:12.635265 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c8f677d0-a3ae-45ce-8b87-3c028857614b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33a46571f6a64b65aa0e37f67cfdae0c', 'request_id': 'req-c8f677d0-a3ae-45ce-8b87-3c028857614b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:12.725435 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c8f677d0-a3ae-45ce-8b87-3c028857614b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33a46571f6a64b65aa0e37f67cfdae0c', 'request_id': 'req-c8f677d0-a3ae-45ce-8b87-3c028857614b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:12.733015 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c8f677d0-a3ae-45ce-8b87-3c028857614b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33a46571f6a64b65aa0e37f67cfdae0c', 'request_id': 'req-c8f677d0-a3ae-45ce-8b87-3c028857614b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:12.737996 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:22:12.744542 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c8f677d0-a3ae-45ce-8b87-3c028857614b admin admin] The token cache is being invalidate because project 33a46571f6a64b65aa0e37f67cfdae0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:22:12.744832 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c8f677d0-a3ae-45ce-8b87-3c028857614b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:12.745491 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c8f677d0-a3ae-45ce-8b87-3c028857614b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:12.747041 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1454/2879] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:22:12 2022] DELETE /identity/v3/projects/33a46571f6a64b65aa0e37f67cfdae0c => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:22:12.751982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2083837-87a2-4391-8ecc-2202c7f4d1f1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:12.755099 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2083837-87a2-4391-8ecc-2202c7f4d1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:12.755099 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2083837-87a2-4391-8ecc-2202c7f4d1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:12.772850 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e2083837-87a2-4391-8ecc-2202c7f4d1f1 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:22:12.774173 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e2083837-87a2-4391-8ecc-2202c7f4d1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:12.835701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1426/2880] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:22:12.843682 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df8818-4ed4-4acc-8205-7f9ef02ff843 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:12.843952 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df8818-4ed4-4acc-8205-7f9ef02ff843 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:12.844233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df8818-4ed4-4acc-8205-7f9ef02ff843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:12.864520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-85df8818-4ed4-4acc-8205-7f9ef02ff843 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:22:12.865750 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-85df8818-4ed4-4acc-8205-7f9ef02ff843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:12.873900 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1455/2881] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:22:12.878138 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2083837-87a2-4391-8ecc-2202c7f4d1f1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:12.888376 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2083837-87a2-4391-8ecc-2202c7f4d1f1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:12.892991 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bb514-6917-4315-b447-78b808b2cbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:12.895880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bb514-6917-4315-b447-78b808b2cbf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:12.895880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bb514-6917-4315-b447-78b808b2cbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:12.895880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bb514-6917-4315-b447-78b808b2cbf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:12.895880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23bb514-6917-4315-b447-78b808b2cbf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:22:12.895880 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23bb514-6917-4315-b447-78b808b2cbf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:12.896964 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23bb514-6917-4315-b447-78b808b2cbf6 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:12.909366 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1427/2882] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:22:12.913836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df8818-4ed4-4acc-8205-7f9ef02ff843 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:12.923331 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df8818-4ed4-4acc-8205-7f9ef02ff843 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:12.928603 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46e010-b06e-47d4-9cfe-14bc245bcb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:12.929223 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46e010-b06e-47d4-9cfe-14bc245bcb98 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:12.929534 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46e010-b06e-47d4-9cfe-14bc245bcb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:12.929812 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46e010-b06e-47d4-9cfe-14bc245bcb98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:12.930472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46e010-b06e-47d4-9cfe-14bc245bcb98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:12.932391 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46e010-b06e-47d4-9cfe-14bc245bcb98 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:13.013827 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1456/2883] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:22:12 2022] POST /identity/v3/projects => generated 391 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:22:13.017737 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bb514-6917-4315-b447-78b808b2cbf6 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:13.029177 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bb514-6917-4315-b447-78b808b2cbf6 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:13.085126 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2b78b-bb57-490c-a5c4-53143dffbede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:13.085764 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2b78b-bb57-490c-a5c4-53143dffbede admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:13.086051 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2b78b-bb57-490c-a5c4-53143dffbede admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:13.086328 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2b78b-bb57-490c-a5c4-53143dffbede admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:13.086956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2b78b-bb57-490c-a5c4-53143dffbede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:13.088859 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2b78b-bb57-490c-a5c4-53143dffbede admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:13.097817 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:22:13.177801 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1428/2884] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:22:13 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:22:13.182582 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46e010-b06e-47d4-9cfe-14bc245bcb98 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:13.194173 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46e010-b06e-47d4-9cfe-14bc245bcb98 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:13.199483 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3a32e-47ea-440c-a900-544b3f677550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:13.200221 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3a32e-47ea-440c-a900-544b3f677550 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:13.200511 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3a32e-47ea-440c-a900-544b3f677550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:13.200789 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3a32e-47ea-440c-a900-544b3f677550 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:13.201436 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3a32e-47ea-440c-a900-544b3f677550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:13.203085 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3a32e-47ea-440c-a900-544b3f677550 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:13.214653 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1457/2885] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:22:13.218811 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2b78b-bb57-490c-a5c4-53143dffbede admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:13.229085 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2b78b-bb57-490c-a5c4-53143dffbede admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:13.233815 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189d550-dd4f-4307-bda8-407209671be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:13.234450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189d550-dd4f-4307-bda8-407209671be4 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:13.234731 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189d550-dd4f-4307-bda8-407209671be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:13.235007 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189d550-dd4f-4307-bda8-407209671be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:13.235729 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189d550-dd4f-4307-bda8-407209671be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:13.237330 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189d550-dd4f-4307-bda8-407209671be4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:13.249463 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1429/2886] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:22:13.254050 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3a32e-47ea-440c-a900-544b3f677550 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:13.264902 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3a32e-47ea-440c-a900-544b3f677550 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:13.269839 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ab844-da1f-420d-adc0-faf8a1343140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:13.270520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ab844-da1f-420d-adc0-faf8a1343140 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:13.270806 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ab844-da1f-420d-adc0-faf8a1343140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:13.271084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ab844-da1f-420d-adc0-faf8a1343140 admin admin] PATH_INFO: `/v3/projects/9ca3eebf211542668e3f84fed4005180/users/41912d17acac46d1882d737ef8a27b02/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:13.285671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ab844-da1f-420d-adc0-faf8a1343140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ca3eebf211542668e3f84fed4005180, user_id=41912d17acac46d1882d737ef8a27b02, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:13.288336 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ab844-da1f-420d-adc0-faf8a1343140 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:13.365837 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1458/2887] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:22:13 2022] PUT /identity/v3/projects/9ca3eebf211542668e3f84fed4005180/users/41912d17acac46d1882d737ef8a27b02/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:22:13.376927 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df7068-6d94-4286-a865-169e111c27f5 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:13.377345 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df7068-6d94-4286-a865-169e111c27f5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:13.377682 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df7068-6d94-4286-a865-169e111c27f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:13.407494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-11df7068-6d94-4286-a865-169e111c27f5 None None] MFA Rules not processed for user `41912d17acac46d1882d737ef8a27b02`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:22:13.408824 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-11df7068-6d94-4286-a865-169e111c27f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:13.473579 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1430/2888] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:22:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:22:13.485371 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ab844-da1f-420d-adc0-faf8a1343140 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:13.490217 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-bf5ab844-da1f-420d-adc0-faf8a1343140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:13.547049 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ab844-da1f-420d-adc0-faf8a1343140 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:13.550856 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967966fd-6586-41e4-bba4-40660fa9ed57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:13.551472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967966fd-6586-41e4-bba4-40660fa9ed57 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:13.551758 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967966fd-6586-41e4-bba4-40660fa9ed57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:13.552063 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967966fd-6586-41e4-bba4-40660fa9ed57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:13.561434 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967966fd-6586-41e4-bba4-40660fa9ed57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:13.562912 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967966fd-6586-41e4-bba4-40660fa9ed57 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:13.571931 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1459/2889] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:22:13 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:22:23.470789 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:22:23.567734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=55,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:22:39.929361 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df7068-6d94-4286-a865-169e111c27f5 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:40.017798 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df7068-6d94-4286-a865-169e111c27f5 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:40.023948 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fcf27-741f-4001-9edc-3c3a79afa190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:40.024967 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fcf27-741f-4001-9edc-3c3a79afa190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:40.025305 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fcf27-741f-4001-9edc-3c3a79afa190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:40.025584 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fcf27-741f-4001-9edc-3c3a79afa190 admin admin] PATH_INFO: `/v3/users/41912d17acac46d1882d737ef8a27b02` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:40.027255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fcf27-741f-4001-9edc-3c3a79afa190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41912d17acac46d1882d737ef8a27b02)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:40.030292 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fcf27-741f-4001-9edc-3c3a79afa190 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:40.272680 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-af4fcf27-741f-4001-9edc-3c3a79afa190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41912d17acac46d1882d737ef8a27b02'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:40.285040 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-af4fcf27-741f-4001-9edc-3c3a79afa190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41912d17acac46d1882d737ef8a27b02'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:40.292003 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-af4fcf27-741f-4001-9edc-3c3a79afa190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41912d17acac46d1882d737ef8a27b02'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:40.382072 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-af4fcf27-741f-4001-9edc-3c3a79afa190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41912d17acac46d1882d737ef8a27b02'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:40.382799 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-af4fcf27-741f-4001-9edc-3c3a79afa190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41912d17acac46d1882d737ef8a27b02'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:40.390471 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1431/2890] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:22:39 2022] DELETE /identity/v3/users/41912d17acac46d1882d737ef8a27b02 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:22:40.395766 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac0628-a187-4b90-8e8b-0ba6c9d8e29a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:40.396602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac0628-a187-4b90-8e8b-0ba6c9d8e29a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:40.396838 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac0628-a187-4b90-8e8b-0ba6c9d8e29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:40.423074 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0eac0628-a187-4b90-8e8b-0ba6c9d8e29a None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:22:40.424429 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0eac0628-a187-4b90-8e8b-0ba6c9d8e29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:40.509467 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1460/2891] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:22:40.525398 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fcf27-741f-4001-9edc-3c3a79afa190 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:40.525398 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-af4fcf27-741f-4001-9edc-3c3a79afa190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:40.587500 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fcf27-741f-4001-9edc-3c3a79afa190 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:40.591222 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33323e02-aa62-40d1-bbe8-6648726edb06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:40.591875 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33323e02-aa62-40d1-bbe8-6648726edb06 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:40.592165 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33323e02-aa62-40d1-bbe8-6648726edb06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:40.592446 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33323e02-aa62-40d1-bbe8-6648726edb06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:40.602002 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33323e02-aa62-40d1-bbe8-6648726edb06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:40.603432 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33323e02-aa62-40d1-bbe8-6648726edb06 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:40.613808 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1432/2892] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:22:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:22:41.116664 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac0628-a187-4b90-8e8b-0ba6c9d8e29a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:41.118177 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0eac0628-a187-4b90-8e8b-0ba6c9d8e29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:41.131373 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac0628-a187-4b90-8e8b-0ba6c9d8e29a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:41.136042 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06442043-73ec-4e39-9c02-3966a26812d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:41.136730 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06442043-73ec-4e39-9c02-3966a26812d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:41.137018 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06442043-73ec-4e39-9c02-3966a26812d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:41.137299 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06442043-73ec-4e39-9c02-3966a26812d2 admin admin] PATH_INFO: `/v3/projects/9ca3eebf211542668e3f84fed4005180` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:41.138585 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06442043-73ec-4e39-9c02-3966a26812d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ca3eebf211542668e3f84fed4005180)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:41.140570 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06442043-73ec-4e39-9c02-3966a26812d2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:41.291753 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-06442043-73ec-4e39-9c02-3966a26812d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ca3eebf211542668e3f84fed4005180', 'request_id': 'req-06442043-73ec-4e39-9c02-3966a26812d2'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:41.370963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-06442043-73ec-4e39-9c02-3966a26812d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ca3eebf211542668e3f84fed4005180', 'request_id': 'req-06442043-73ec-4e39-9c02-3966a26812d2'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:41.379350 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-06442043-73ec-4e39-9c02-3966a26812d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ca3eebf211542668e3f84fed4005180', 'request_id': 'req-06442043-73ec-4e39-9c02-3966a26812d2'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:41.390690 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-06442043-73ec-4e39-9c02-3966a26812d2 admin admin] The token cache is being invalidate because project 9ca3eebf211542668e3f84fed4005180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:22:41.390982 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-06442043-73ec-4e39-9c02-3966a26812d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:41.391944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-06442043-73ec-4e39-9c02-3966a26812d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:22:41.395489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1461/2893] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:22:41 2022] DELETE /identity/v3/projects/9ca3eebf211542668e3f84fed4005180 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:22:41.399653 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ff4b2-db43-4237-bcc7-39728e0bc233 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:41.400256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ff4b2-db43-4237-bcc7-39728e0bc233 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:41.400256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ff4b2-db43-4237-bcc7-39728e0bc233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:41.421053 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-177ff4b2-db43-4237-bcc7-39728e0bc233 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:22:41.422383 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-177ff4b2-db43-4237-bcc7-39728e0bc233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:41.441536 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:22:41.484318 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1433/2894] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:22:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:22:41.492103 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e027d9-5ccb-4109-a9b3-c4bad2b52757 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:41.492585 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e027d9-5ccb-4109-a9b3-c4bad2b52757 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:41.492689 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e027d9-5ccb-4109-a9b3-c4bad2b52757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:41.515007 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-c4e027d9-5ccb-4109-a9b3-c4bad2b52757 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:22:41.516209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c4e027d9-5ccb-4109-a9b3-c4bad2b52757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:41.525132 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1462/2895] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:22:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:22:41.529359 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ff4b2-db43-4237-bcc7-39728e0bc233 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:41.539621 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ff4b2-db43-4237-bcc7-39728e0bc233 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:41.545645 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa822f6-47b1-4975-b06f-8fe76accada0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:41.546403 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa822f6-47b1-4975-b06f-8fe76accada0 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:41.546762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa822f6-47b1-4975-b06f-8fe76accada0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:41.547113 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa822f6-47b1-4975-b06f-8fe76accada0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:41.547852 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa822f6-47b1-4975-b06f-8fe76accada0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 12:22:41.548273 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa822f6-47b1-4975-b06f-8fe76accada0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:41.550181 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa822f6-47b1-4975-b06f-8fe76accada0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:41.572252 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1434/2896] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 14 12:22:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 12:22:41.573540 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e027d9-5ccb-4109-a9b3-c4bad2b52757 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:41.585056 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e027d9-5ccb-4109-a9b3-c4bad2b52757 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:41.591038 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8688f-d3ad-4d25-b16a-f4b27cb8e0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:41.591722 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8688f-d3ad-4d25-b16a-f4b27cb8e0ec admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:41.592516 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8688f-d3ad-4d25-b16a-f4b27cb8e0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:41.592790 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8688f-d3ad-4d25-b16a-f4b27cb8e0ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:41.593437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f8688f-d3ad-4d25-b16a-f4b27cb8e0ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:41.595367 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f8688f-d3ad-4d25-b16a-f4b27cb8e0ec admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:41.714525 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1463/2897] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:22:41 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:22:41.715810 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa822f6-47b1-4975-b06f-8fe76accada0 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:41.726357 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa822f6-47b1-4975-b06f-8fe76accada0 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:41.789037 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e953060-23ed-4792-854d-73109ea96151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:41.789669 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e953060-23ed-4792-854d-73109ea96151 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:41.789983 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e953060-23ed-4792-854d-73109ea96151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:41.790258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e953060-23ed-4792-854d-73109ea96151 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:41.790886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e953060-23ed-4792-854d-73109ea96151 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:41.792832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e953060-23ed-4792-854d-73109ea96151 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:41.874775 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1435/2898] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:22:41 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:22:41.879101 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8688f-d3ad-4d25-b16a-f4b27cb8e0ec admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:41.890736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8688f-d3ad-4d25-b16a-f4b27cb8e0ec admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:41.895679 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0353b-b58a-444f-98e1-aca3cd678188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:41.905241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0353b-b58a-444f-98e1-aca3cd678188 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:41.905241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0353b-b58a-444f-98e1-aca3cd678188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:41.905241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0353b-b58a-444f-98e1-aca3cd678188 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:41.905241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0353b-b58a-444f-98e1-aca3cd678188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:41.905241 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0353b-b58a-444f-98e1-aca3cd678188 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:41.910246 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1464/2899] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:22:41.914485 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e953060-23ed-4792-854d-73109ea96151 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:41.924681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e953060-23ed-4792-854d-73109ea96151 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:41.929657 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9429c38-7814-4d75-8c51-9424c7dca9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:41.930294 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9429c38-7814-4d75-8c51-9424c7dca9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:41.930582 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9429c38-7814-4d75-8c51-9424c7dca9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:41.930864 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9429c38-7814-4d75-8c51-9424c7dca9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:41.931537 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9429c38-7814-4d75-8c51-9424c7dca9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:41.933187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9429c38-7814-4d75-8c51-9424c7dca9f9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:41.944661 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1436/2900] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:22:41.948976 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0353b-b58a-444f-98e1-aca3cd678188 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:41.960163 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0353b-b58a-444f-98e1-aca3cd678188 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:41.965021 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f864f71-5373-4d07-b810-7017920c1999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:41.965744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f864f71-5373-4d07-b810-7017920c1999 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:41.966053 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f864f71-5373-4d07-b810-7017920c1999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:41.966328 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f864f71-5373-4d07-b810-7017920c1999 admin admin] PATH_INFO: `/v3/projects/b9a967a1f3444a1e8b7f4321722b5949/users/6e9703bf61d14dc4ac52a7c50fb8390e/roles/d96ee0e293244ae6a04e8ce652aea68e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:41.981791 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f864f71-5373-4d07-b810-7017920c1999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a967a1f3444a1e8b7f4321722b5949, user_id=6e9703bf61d14dc4ac52a7c50fb8390e, role_id=d96ee0e293244ae6a04e8ce652aea68e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:41.985283 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f864f71-5373-4d07-b810-7017920c1999 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:42.063017 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1465/2901] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:22:41 2022] PUT /identity/v3/projects/b9a967a1f3444a1e8b7f4321722b5949/users/6e9703bf61d14dc4ac52a7c50fb8390e/roles/d96ee0e293244ae6a04e8ce652aea68e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:22:42.068583 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15feb03b-0145-4233-bb4f-afb4c32ce956 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:42.068883 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15feb03b-0145-4233-bb4f-afb4c32ce956 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:42.069165 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15feb03b-0145-4233-bb4f-afb4c32ce956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:42.089089 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-15feb03b-0145-4233-bb4f-afb4c32ce956 None None] MFA Rules not processed for user `6c2df9ec255e48ec9c7b4291f711f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:22:42.090550 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-15feb03b-0145-4233-bb4f-afb4c32ce956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:42.152141 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1437/2902] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:22:42.162975 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f864f71-5373-4d07-b810-7017920c1999 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:42.168066 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7f864f71-5373-4d07-b810-7017920c1999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:42.187483 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:22:42.224250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f864f71-5373-4d07-b810-7017920c1999 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:42.228157 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660feb4-b4b5-456a-b789-05fccc999ab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:42.228763 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660feb4-b4b5-456a-b789-05fccc999ab5 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:42.229042 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660feb4-b4b5-456a-b789-05fccc999ab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:42.229315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660feb4-b4b5-456a-b789-05fccc999ab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:42.240117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4660feb4-b4b5-456a-b789-05fccc999ab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:42.241527 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4660feb4-b4b5-456a-b789-05fccc999ab5 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:42.251248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1466/2903] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:22:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:22:48.180512 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe41c5-77f4-4030-970c-29de409135e5 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:48.181073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe41c5-77f4-4030-970c-29de409135e5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:48.181211 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe41c5-77f4-4030-970c-29de409135e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:48.222753 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-69fe41c5-77f4-4030-970c-29de409135e5 None None] MFA Rules not processed for user `6e9703bf61d14dc4ac52a7c50fb8390e`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:22:48.224786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-69fe41c5-77f4-4030-970c-29de409135e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:48.264216 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:22:48.337070 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1438/2904] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:22:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:22:48.345501 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660feb4-b4b5-456a-b789-05fccc999ab5 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:48.358329 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660feb4-b4b5-456a-b789-05fccc999ab5 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:48.370886 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763bf05b-4122-4859-afce-9fd39374bfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:48.382882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bf05b-4122-4859-afce-9fd39374bfb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:48.382882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bf05b-4122-4859-afce-9fd39374bfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:48.382882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bf05b-4122-4859-afce-9fd39374bfb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:48.382882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763bf05b-4122-4859-afce-9fd39374bfb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:48.382882 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763bf05b-4122-4859-afce-9fd39374bfb4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:48.505295 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1467/2905] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 14 12:22:48 2022] POST /identity/v3/projects => generated 383 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:22:48.509672 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe41c5-77f4-4030-970c-29de409135e5 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:48.522486 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe41c5-77f4-4030-970c-29de409135e5 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:48.592286 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f5243-ff1e-4751-b763-57820b9c4d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:48.592930 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f5243-ff1e-4751-b763-57820b9c4d16 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:48.593218 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f5243-ff1e-4751-b763-57820b9c4d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:48.593499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f5243-ff1e-4751-b763-57820b9c4d16 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:48.594183 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f5243-ff1e-4751-b763-57820b9c4d16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:48.596298 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f5243-ff1e-4751-b763-57820b9c4d16 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:48.685650 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1439/2906] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 14 12:22:48 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 12:22:48.689729 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763bf05b-4122-4859-afce-9fd39374bfb4 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:48.700761 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763bf05b-4122-4859-afce-9fd39374bfb4 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:48.706805 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bc9e1-de01-4db7-84ab-c146d3416ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:48.707441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bc9e1-de01-4db7-84ab-c146d3416ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:48.707731 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bc9e1-de01-4db7-84ab-c146d3416ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:48.708004 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bc9e1-de01-4db7-84ab-c146d3416ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:48.708659 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921bc9e1-de01-4db7-84ab-c146d3416ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:48.710298 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921bc9e1-de01-4db7-84ab-c146d3416ca6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:48.726130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1468/2907] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 12:22:48.730692 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f5243-ff1e-4751-b763-57820b9c4d16 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:48.741658 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f5243-ff1e-4751-b763-57820b9c4d16 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:48.746388 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbe447-ca0b-4937-a0cd-81b63389cef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:48.747018 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbe447-ca0b-4937-a0cd-81b63389cef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:48.747301 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbe447-ca0b-4937-a0cd-81b63389cef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:48.747683 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbe447-ca0b-4937-a0cd-81b63389cef8 admin admin] PATH_INFO: `/v3/projects/fa63592046c24d13a72f02371ee4f417/users/001a3299f26643c6b9c13e80a21f3eba/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:48.764598 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbe447-ca0b-4937-a0cd-81b63389cef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa63592046c24d13a72f02371ee4f417, user_id=001a3299f26643c6b9c13e80a21f3eba, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:48.767428 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbe447-ca0b-4937-a0cd-81b63389cef8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:48.853938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1440/2908] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:22:48 2022] PUT /identity/v3/projects/fa63592046c24d13a72f02371ee4f417/users/001a3299f26643c6b9c13e80a21f3eba/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:22:48.855122 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bc9e1-de01-4db7-84ab-c146d3416ca6 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:48.869918 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bc9e1-de01-4db7-84ab-c146d3416ca6 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:48.931300 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497b407-d7c5-451b-93e5-c42862365f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:48.931924 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497b407-d7c5-451b-93e5-c42862365f46 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:48.932209 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497b407-d7c5-451b-93e5-c42862365f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:48.932489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497b407-d7c5-451b-93e5-c42862365f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:48.933137 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497b407-d7c5-451b-93e5-c42862365f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:48.934770 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497b407-d7c5-451b-93e5-c42862365f46 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:48.945771 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1469/2909] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 14 12:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 12:22:48.950028 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbe447-ca0b-4937-a0cd-81b63389cef8 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:48.965800 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbe447-ca0b-4937-a0cd-81b63389cef8 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:48.972423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0691656-056a-4c49-9ccc-2e241490d58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:48.974513 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0691656-056a-4c49-9ccc-2e241490d58d admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:48.975994 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0691656-056a-4c49-9ccc-2e241490d58d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:48.977182 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0691656-056a-4c49-9ccc-2e241490d58d admin admin] PATH_INFO: `/v3/projects/fa63592046c24d13a72f02371ee4f417/users/001a3299f26643c6b9c13e80a21f3eba/roles/4a031234e1ef4fdf8ce238fbdd96fc7d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:48.979520 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0691656-056a-4c49-9ccc-2e241490d58d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa63592046c24d13a72f02371ee4f417, user_id=001a3299f26643c6b9c13e80a21f3eba, role_id=4a031234e1ef4fdf8ce238fbdd96fc7d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:48.982450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0691656-056a-4c49-9ccc-2e241490d58d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:48.998469 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1441/2910] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 14 12:22:48 2022] PUT /identity/v3/projects/fa63592046c24d13a72f02371ee4f417/users/001a3299f26643c6b9c13e80a21f3eba/roles/4a031234e1ef4fdf8ce238fbdd96fc7d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 12:22:53.414566 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3fdad-37fb-4454-81df-90400cd359bb None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:53.415402 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3fdad-37fb-4454-81df-90400cd359bb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:53.415402 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3fdad-37fb-4454-81df-90400cd359bb None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:53.418434 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1470/2911] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 14 12:22:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:22:53.435333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdf64a-27a3-412d-b04a-64696af93799 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:53.436033 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdf64a-27a3-412d-b04a-64696af93799 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:53.436033 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdf64a-27a3-412d-b04a-64696af93799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:53.491874 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f0bdf64a-27a3-412d-b04a-64696af93799 None None] MFA Rules not processed for user `13f1840d92d94c3ebd16000a4889c377`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:22:53.493228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f0bdf64a-27a3-412d-b04a-64696af93799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:53.584748 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:22:53.598168 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1442/2912] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 14 12:22:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:22:53.615489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3fdad-37fb-4454-81df-90400cd359bb None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:53.641707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3fdad-37fb-4454-81df-90400cd359bb None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:53.652162 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12051b-6b74-4347-8a78-35472f1c5f09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:53.652993 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12051b-6b74-4347-8a78-35472f1c5f09 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:53.654493 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12051b-6b74-4347-8a78-35472f1c5f09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:53.654907 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12051b-6b74-4347-8a78-35472f1c5f09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:53.665531 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a12051b-6b74-4347-8a78-35472f1c5f09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:53.668080 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a12051b-6b74-4347-8a78-35472f1c5f09 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:53.682906 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1471/2913] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:22:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:22:54.068930 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdf64a-27a3-412d-b04a-64696af93799 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:54.070549 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f0bdf64a-27a3-412d-b04a-64696af93799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:54.197734 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdf64a-27a3-412d-b04a-64696af93799 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:54.208684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba93e1-33b0-48d2-a818-559c0a092b2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:54.209691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba93e1-33b0-48d2-a818-559c0a092b2d service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:54.214325 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba93e1-33b0-48d2-a818-559c0a092b2d service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:54.214325 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba93e1-33b0-48d2-a818-559c0a092b2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:54.214325 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c5ba93e1-33b0-48d2-a818-559c0a092b2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:54.232952 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba93e1-33b0-48d2-a818-559c0a092b2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:54.236065 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba93e1-33b0-48d2-a818-559c0a092b2d service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:54.259120 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1443/2914] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:22:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:22:54.813838 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b9430-237d-476a-a8cd-8d95c9e83965 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:54.815262 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b9430-237d-476a-a8cd-8d95c9e83965 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:54.816307 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b9430-237d-476a-a8cd-8d95c9e83965 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:54.827834 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1472/2915] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 14 12:22:54 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 12:22:54.837511 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f91705-623f-4742-8517-9c58c92ebeea None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:54.842600 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f91705-623f-4742-8517-9c58c92ebeea None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:54.843466 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f91705-623f-4742-8517-9c58c92ebeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:54.933723 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b1f91705-623f-4742-8517-9c58c92ebeea None None] MFA Rules not processed for user `38b093bc4c1d4ebcbd2a6bb16b70c029`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:22:54.935971 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b1f91705-623f-4742-8517-9c58c92ebeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:54.948275 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1444/2916] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 14 12:22:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:22:54.960342 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b9430-237d-476a-a8cd-8d95c9e83965 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:54.979921 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b9430-237d-476a-a8cd-8d95c9e83965 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:54.984357 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38479878-1651-4508-b9ad-a61be31eae69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:54.984989 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38479878-1651-4508-b9ad-a61be31eae69 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:54.985272 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38479878-1651-4508-b9ad-a61be31eae69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:54.985549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38479878-1651-4508-b9ad-a61be31eae69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:55.002407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38479878-1651-4508-b9ad-a61be31eae69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:55.002407 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38479878-1651-4508-b9ad-a61be31eae69 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:55.016674 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1473/2917] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:22:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:22:55.863118 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb110e-206f-4699-92f1-49e4ab669d28 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:55.863691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb110e-206f-4699-92f1-49e4ab669d28 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:55.863691 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb110e-206f-4699-92f1-49e4ab669d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:55.895659 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-cceb110e-206f-4699-92f1-49e4ab669d28 None None] MFA Rules not processed for user `001a3299f26643c6b9c13e80a21f3eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:22:55.897064 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cceb110e-206f-4699-92f1-49e4ab669d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:56.008596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1445/2918] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:22:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:22:56.026318 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38479878-1651-4508-b9ad-a61be31eae69 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:56.029248 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-38479878-1651-4508-b9ad-a61be31eae69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:56.133814 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38479878-1651-4508-b9ad-a61be31eae69 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:56.137821 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1bc9a-d59e-4e4d-9221-84ff072e4054 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b05d0e65e9645f3810a6d96c8fed9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:56.138515 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1bc9a-d59e-4e4d-9221-84ff072e4054 service glance] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:56.138799 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1bc9a-d59e-4e4d-9221-84ff072e4054 service glance] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:56.139076 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1bc9a-d59e-4e4d-9221-84ff072e4054 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:56.155372 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:22:56.213572 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1bc9a-d59e-4e4d-9221-84ff072e4054 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:56.215099 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1bc9a-d59e-4e4d-9221-84ff072e4054 service glance] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:56.225645 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1474/2919] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:22:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:22:56.608441 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3c3f9-abe7-4818-b44c-f147e807b92f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:56.608741 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3c3f9-abe7-4818-b44c-f147e807b92f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:56.609020 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3c3f9-abe7-4818-b44c-f147e807b92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:56.636899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-43d3c3f9-abe7-4818-b44c-f147e807b92f None None] MFA Rules not processed for user `d76a164ed7e14d9eb2db016b68a84d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:22:56.638518 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-43d3c3f9-abe7-4818-b44c-f147e807b92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:56.694703 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:22:56.717459 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1446/2920] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 14 12:22:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:22:56.730381 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1bc9a-d59e-4e4d-9221-84ff072e4054 service glance] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:22:56.732043 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-75b1bc9a-d59e-4e4d-9221-84ff072e4054 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:22:56.836167 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1bc9a-d59e-4e4d-9221-84ff072e4054 service glance] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:22:56.843809 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c2086-2b22-4286-884b-3059753142b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1047c7d18c1466d989cd895117cd807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:22:56.844437 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c2086-2b22-4286-884b-3059753142b8 service swift] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:22:56.844718 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c2086-2b22-4286-884b-3059753142b8 service swift] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:22:56.845016 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c2086-2b22-4286-884b-3059753142b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:22:56.860654 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c2086-2b22-4286-884b-3059753142b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:22:56.869427 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c2086-2b22-4286-884b-3059753142b8 service swift] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:22:56.872391 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1475/2921] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 14 12:22:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 12:23:06.724744 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:23:06.868278 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=53 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:23:08.118450 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60942d50-1ab1-4903-90ae-1e3a3c9ebd5b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:23:08.118995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60942d50-1ab1-4903-90ae-1e3a3c9ebd5b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:23:08.118995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60942d50-1ab1-4903-90ae-1e3a3c9ebd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:23:08.139012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-60942d50-1ab1-4903-90ae-1e3a3c9ebd5b None None] MFA Rules not processed for user `6e9703bf61d14dc4ac52a7c50fb8390e`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:23:08.140256 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-60942d50-1ab1-4903-90ae-1e3a3c9ebd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:23:08.147454 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1447/2922] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:23:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:23:08.408787 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c2086-2b22-4286-884b-3059753142b8 service swift] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:23:08.418187 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c2086-2b22-4286-884b-3059753142b8 service swift] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:23:08.422366 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd847a-6c22-4c74-84d9-f13edf7e1a75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:23:08.422963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd847a-6c22-4c74-84d9-f13edf7e1a75 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:23:08.423250 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd847a-6c22-4c74-84d9-f13edf7e1a75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:23:08.423509 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd847a-6c22-4c74-84d9-f13edf7e1a75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:23:08.425212 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e2dd847a-6c22-4c74-84d9-f13edf7e1a75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:23:08.435489 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd847a-6c22-4c74-84d9-f13edf7e1a75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:23:08.436899 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd847a-6c22-4c74-84d9-f13edf7e1a75 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:23:08.446009 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1476/2923] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:23:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:23:12.457238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37de44-69c9-4815-954f-e2338a20b9ee None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:23:12.457768 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37de44-69c9-4815-954f-e2338a20b9ee None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:23:12.457933 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37de44-69c9-4815-954f-e2338a20b9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:23:12.478982 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3a37de44-69c9-4815-954f-e2338a20b9ee None None] MFA Rules not processed for user `d76a164ed7e14d9eb2db016b68a84d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:23:12.480230 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3a37de44-69c9-4815-954f-e2338a20b9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:23:12.493922 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1448/2924] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 14 12:23:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:23:12.519054 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd847a-6c22-4c74-84d9-f13edf7e1a75 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:23:12.522130 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd847a-6c22-4c74-84d9-f13edf7e1a75 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:23:12.528609 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95030cc0-4653-406a-8d53-116ac86de123 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1047c7d18c1466d989cd895117cd807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:23:12.529364 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95030cc0-4653-406a-8d53-116ac86de123 service swift] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:23:12.529745 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95030cc0-4653-406a-8d53-116ac86de123 service swift] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:23:12.530315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95030cc0-4653-406a-8d53-116ac86de123 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:23:12.539773 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95030cc0-4653-406a-8d53-116ac86de123 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:23:12.541545 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95030cc0-4653-406a-8d53-116ac86de123 service swift] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:23:12.551465 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1477/2925] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 14 12:23:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 12:23:22.472996 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=6 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:23:22.547888 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=12 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:23:24.348850 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37de44-69c9-4815-954f-e2338a20b9ee None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:23:24.382556 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37de44-69c9-4815-954f-e2338a20b9ee None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:23:24.389684 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189a79f-3206-46ac-be60-9e8e2c6cd894 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:23:24.390404 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189a79f-3206-46ac-be60-9e8e2c6cd894 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:23:24.390688 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189a79f-3206-46ac-be60-9e8e2c6cd894 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:23:24.390950 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189a79f-3206-46ac-be60-9e8e2c6cd894 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:23:24.392762 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b189a79f-3206-46ac-be60-9e8e2c6cd894 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:23:24.427596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189a79f-3206-46ac-be60-9e8e2c6cd894 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:23:24.429237 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189a79f-3206-46ac-be60-9e8e2c6cd894 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:23:24.440590 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1449/2926] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:23:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:23:31.003576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b4cdd-8eb1-4e13-85f1-672edd78bbc9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:23:31.004084 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b4cdd-8eb1-4e13-85f1-672edd78bbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:23:31.004262 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b4cdd-8eb1-4e13-85f1-672edd78bbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:23:31.025159 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1f0b4cdd-8eb1-4e13-85f1-672edd78bbc9 None None] MFA Rules not processed for user `33f480ef1b8b49df83276a66f445ae85`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:23:31.026420 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1f0b4cdd-8eb1-4e13-85f1-672edd78bbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:23:31.091031 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1478/2927] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:23:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:23:34.436102 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=22 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:23:41.079337 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=25 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:23:49.478832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189a79f-3206-46ac-be60-9e8e2c6cd894 service nova] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:23:49.488400 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189a79f-3206-46ac-be60-9e8e2c6cd894 service nova] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:23:49.492154 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017abeed-de18-472a-a06f-cd781b678a60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:23:49.492755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017abeed-de18-472a-a06f-cd781b678a60 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:23:49.493042 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017abeed-de18-472a-a06f-cd781b678a60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:23:49.493330 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017abeed-de18-472a-a06f-cd781b678a60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:23:49.503460 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017abeed-de18-472a-a06f-cd781b678a60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:23:49.504846 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017abeed-de18-472a-a06f-cd781b678a60 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:23:49.515394 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1450/2928] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:23:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:23:59.510573 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=6 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:25:12.537905 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b4cdd-8eb1-4e13-85f1-672edd78bbc9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:25:12.539411 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1f0b4cdd-8eb1-4e13-85f1-672edd78bbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:25:12.638963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b4cdd-8eb1-4e13-85f1-672edd78bbc9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:25:12.646622 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5844c99-7473-4359-9c07-50385b0986f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca10dd7ed714edb8b87017deecc41b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:25:12.647261 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5844c99-7473-4359-9c07-50385b0986f5 service placement] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:25:12.647550 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5844c99-7473-4359-9c07-50385b0986f5 service placement] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:25:12.647827 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5844c99-7473-4359-9c07-50385b0986f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:25:12.650003 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c5844c99-7473-4359-9c07-50385b0986f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:25:12.663477 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5844c99-7473-4359-9c07-50385b0986f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:25:12.665117 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5844c99-7473-4359-9c07-50385b0986f5 service placement] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:25:12.676742 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1479/2929] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:25:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:25:16.545385 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017abeed-de18-472a-a06f-cd781b678a60 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:25:16.546839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-017abeed-de18-472a-a06f-cd781b678a60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:25:16.582870 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017abeed-de18-472a-a06f-cd781b678a60 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:25:16.586766 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459279ac-5722-44c1-aeb5-76d6aaa11d4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:25:16.587375 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459279ac-5722-44c1-aeb5-76d6aaa11d4c service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:25:16.587660 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459279ac-5722-44c1-aeb5-76d6aaa11d4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:25:16.587940 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459279ac-5722-44c1-aeb5-76d6aaa11d4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:25:16.590248 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-459279ac-5722-44c1-aeb5-76d6aaa11d4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:25:16.652644 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459279ac-5722-44c1-aeb5-76d6aaa11d4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:25:16.654158 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459279ac-5722-44c1-aeb5-76d6aaa11d4c service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:25:16.664635 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1451/2930] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:25:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:25:22.671637 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=41 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:25:24.630961 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5844c99-7473-4359-9c07-50385b0986f5 service placement] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:25:24.641849 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5844c99-7473-4359-9c07-50385b0986f5 service placement] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:25:24.646231 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13244b1b-ff55-493d-beae-9e146f682a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:25:24.646839 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13244b1b-ff55-493d-beae-9e146f682a86 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:25:24.647129 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13244b1b-ff55-493d-beae-9e146f682a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:25:24.647410 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13244b1b-ff55-493d-beae-9e146f682a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:25:24.649219 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-13244b1b-ff55-493d-beae-9e146f682a86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:25:24.761318 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13244b1b-ff55-493d-beae-9e146f682a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:25:24.762698 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13244b1b-ff55-493d-beae-9e146f682a86 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:25:24.781928 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1480/2931] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:25:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:25:26.659606 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=37 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:25:28.430755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459279ac-5722-44c1-aeb5-76d6aaa11d4c service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:25:28.444904 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459279ac-5722-44c1-aeb5-76d6aaa11d4c service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:25:28.451835 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19883cc-8446-47e4-bd80-c67b4970d5e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca10dd7ed714edb8b87017deecc41b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:25:28.453987 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19883cc-8446-47e4-bd80-c67b4970d5e7 service placement] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:25:28.454752 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19883cc-8446-47e4-bd80-c67b4970d5e7 service placement] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:25:28.454752 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19883cc-8446-47e4-bd80-c67b4970d5e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:25:28.456911 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c19883cc-8446-47e4-bd80-c67b4970d5e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:25:28.473924 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19883cc-8446-47e4-bd80-c67b4970d5e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:25:28.475543 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19883cc-8446-47e4-bd80-c67b4970d5e7 service placement] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:25:28.487318 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1452/2932] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:25:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:25:32.557588 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13244b1b-ff55-493d-beae-9e146f682a86 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:25:32.569405 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13244b1b-ff55-493d-beae-9e146f682a86 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:25:32.575535 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06947292-e809-42bd-89e9-9e14cd40dc18 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca10dd7ed714edb8b87017deecc41b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:25:32.576147 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06947292-e809-42bd-89e9-9e14cd40dc18 service placement] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:25:32.576430 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06947292-e809-42bd-89e9-9e14cd40dc18 service placement] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:25:32.576714 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06947292-e809-42bd-89e9-9e14cd40dc18 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:25:32.578453 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-06947292-e809-42bd-89e9-9e14cd40dc18 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:25:32.591734 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06947292-e809-42bd-89e9-9e14cd40dc18 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:25:32.595786 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06947292-e809-42bd-89e9-9e14cd40dc18 service placement] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:25:32.605914 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1481/2933] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:25:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:25:32.690095 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19883cc-8446-47e4-bd80-c67b4970d5e7 service placement] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:25:32.692238 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c19883cc-8446-47e4-bd80-c67b4970d5e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:25:32.708547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19883cc-8446-47e4-bd80-c67b4970d5e7 service placement] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:25:32.713192 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689e7a8-87a8-442b-9469-95bee3ff1063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:25:32.713839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689e7a8-87a8-442b-9469-95bee3ff1063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:25:32.714271 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689e7a8-87a8-442b-9469-95bee3ff1063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:25:32.714561 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689e7a8-87a8-442b-9469-95bee3ff1063 admin admin] PATH_INFO: `/v3/users/3b11628831fe4f50a7f9997c1f213a71` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:25:32.715701 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689e7a8-87a8-442b-9469-95bee3ff1063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b11628831fe4f50a7f9997c1f213a71)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:25:32.717760 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689e7a8-87a8-442b-9469-95bee3ff1063 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:25:32.947217 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3689e7a8-87a8-442b-9469-95bee3ff1063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b11628831fe4f50a7f9997c1f213a71'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:32.962073 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3689e7a8-87a8-442b-9469-95bee3ff1063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b11628831fe4f50a7f9997c1f213a71'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:32.970071 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3689e7a8-87a8-442b-9469-95bee3ff1063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b11628831fe4f50a7f9997c1f213a71'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:33.048938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3689e7a8-87a8-442b-9469-95bee3ff1063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b11628831fe4f50a7f9997c1f213a71'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:33.049656 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3689e7a8-87a8-442b-9469-95bee3ff1063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b11628831fe4f50a7f9997c1f213a71'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:33.056522 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1453/2934] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:25:32 2022] DELETE /identity/v3/users/3b11628831fe4f50a7f9997c1f213a71 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:25:33.357236 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06947292-e809-42bd-89e9-9e14cd40dc18 service placement] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:25:33.370348 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06947292-e809-42bd-89e9-9e14cd40dc18 service placement] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:25:33.441983 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949475ad-c889-44d2-bee0-141c139a7819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:25:33.442667 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949475ad-c889-44d2-bee0-141c139a7819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:25:33.442956 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949475ad-c889-44d2-bee0-141c139a7819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:25:33.443233 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949475ad-c889-44d2-bee0-141c139a7819 admin admin] PATH_INFO: `/v3/projects/9204365716924bbebdc0d90ecc13d01e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:25:33.444387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949475ad-c889-44d2-bee0-141c139a7819 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9204365716924bbebdc0d90ecc13d01e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:25:33.446404 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949475ad-c889-44d2-bee0-141c139a7819 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:25:33.584741 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-949475ad-c889-44d2-bee0-141c139a7819 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9204365716924bbebdc0d90ecc13d01e', 'request_id': 'req-949475ad-c889-44d2-bee0-141c139a7819'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:33.588559 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:25:33.666432 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-949475ad-c889-44d2-bee0-141c139a7819 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9204365716924bbebdc0d90ecc13d01e', 'request_id': 'req-949475ad-c889-44d2-bee0-141c139a7819'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:33.674082 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-949475ad-c889-44d2-bee0-141c139a7819 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9204365716924bbebdc0d90ecc13d01e', 'request_id': 'req-949475ad-c889-44d2-bee0-141c139a7819'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:33.685946 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-949475ad-c889-44d2-bee0-141c139a7819 admin admin] The token cache is being invalidate because project 9204365716924bbebdc0d90ecc13d01e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:25:33.686239 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-949475ad-c889-44d2-bee0-141c139a7819 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:33.686919 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-949475ad-c889-44d2-bee0-141c139a7819 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:33.688422 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1482/2935] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:25:33 2022] DELETE /identity/v3/projects/9204365716924bbebdc0d90ecc13d01e => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:25:33.693187 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689e7a8-87a8-442b-9469-95bee3ff1063 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:25:33.694664 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3689e7a8-87a8-442b-9469-95bee3ff1063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:25:33.708255 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689e7a8-87a8-442b-9469-95bee3ff1063 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:25:33.889557 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620ad3e-8fcd-44dd-97eb-65ceba803871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:25:33.890227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620ad3e-8fcd-44dd-97eb-65ceba803871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:25:33.890506 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620ad3e-8fcd-44dd-97eb-65ceba803871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:25:33.890787 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620ad3e-8fcd-44dd-97eb-65ceba803871 admin admin] PATH_INFO: `/v3/users/a75421bca5c64ec2a40968085874ed60` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:25:33.892215 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620ad3e-8fcd-44dd-97eb-65ceba803871 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a75421bca5c64ec2a40968085874ed60)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:25:33.894886 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620ad3e-8fcd-44dd-97eb-65ceba803871 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:25:34.094194 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c620ad3e-8fcd-44dd-97eb-65ceba803871 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a75421bca5c64ec2a40968085874ed60'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:34.107316 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:25:34.110624 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c620ad3e-8fcd-44dd-97eb-65ceba803871 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a75421bca5c64ec2a40968085874ed60'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:34.115727 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c620ad3e-8fcd-44dd-97eb-65ceba803871 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a75421bca5c64ec2a40968085874ed60'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:34.180503 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c620ad3e-8fcd-44dd-97eb-65ceba803871 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a75421bca5c64ec2a40968085874ed60'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:34.181128 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c620ad3e-8fcd-44dd-97eb-65ceba803871 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a75421bca5c64ec2a40968085874ed60'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:34.189104 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1454/2936] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:25:33 2022] DELETE /identity/v3/users/a75421bca5c64ec2a40968085874ed60 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:25:34.502496 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949475ad-c889-44d2-bee0-141c139a7819 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:25:34.504087 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-949475ad-c889-44d2-bee0-141c139a7819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:25:34.518116 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949475ad-c889-44d2-bee0-141c139a7819 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:25:34.582601 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83091b-e8c1-4c94-8e40-7c9cfb632c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:25:34.583387 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83091b-e8c1-4c94-8e40-7c9cfb632c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:25:34.583671 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83091b-e8c1-4c94-8e40-7c9cfb632c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:25:34.583944 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83091b-e8c1-4c94-8e40-7c9cfb632c92 admin admin] PATH_INFO: `/v3/projects/c1f845252ff945d0bd5cab90f3405462` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:25:34.585086 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83091b-e8c1-4c94-8e40-7c9cfb632c92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f845252ff945d0bd5cab90f3405462)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:25:34.587049 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83091b-e8c1-4c94-8e40-7c9cfb632c92 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:25:34.732230 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1c83091b-e8c1-4c94-8e40-7c9cfb632c92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f845252ff945d0bd5cab90f3405462', 'request_id': 'req-1c83091b-e8c1-4c94-8e40-7c9cfb632c92'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:34.826418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1c83091b-e8c1-4c94-8e40-7c9cfb632c92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f845252ff945d0bd5cab90f3405462', 'request_id': 'req-1c83091b-e8c1-4c94-8e40-7c9cfb632c92'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:34.834494 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1c83091b-e8c1-4c94-8e40-7c9cfb632c92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f845252ff945d0bd5cab90f3405462', 'request_id': 'req-1c83091b-e8c1-4c94-8e40-7c9cfb632c92'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:34.845586 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1c83091b-e8c1-4c94-8e40-7c9cfb632c92 admin admin] The token cache is being invalidate because project c1f845252ff945d0bd5cab90f3405462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:25:34.845876 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1c83091b-e8c1-4c94-8e40-7c9cfb632c92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:34.846548 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1c83091b-e8c1-4c94-8e40-7c9cfb632c92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:25:34.848045 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1483/2937] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:25:34 2022] DELETE /identity/v3/projects/c1f845252ff945d0bd5cab90f3405462 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:25:44.243280 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:25:44.842173 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=57,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:26:00.902254 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620ad3e-8fcd-44dd-97eb-65ceba803871 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:26:00.904339 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c620ad3e-8fcd-44dd-97eb-65ceba803871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:26:00.990932 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620ad3e-8fcd-44dd-97eb-65ceba803871 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:26:01.039445 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e083f-d181-4123-9e83-69ca0ef73b29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c7ff726e443e89da94f282b5cddab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:26:01.040048 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e083f-d181-4123-9e83-69ca0ef73b29 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:26:01.040326 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e083f-d181-4123-9e83-69ca0ef73b29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:26:01.040596 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e083f-d181-4123-9e83-69ca0ef73b29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:26:01.042864 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-118e083f-d181-4123-9e83-69ca0ef73b29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:26:01.101494 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e083f-d181-4123-9e83-69ca0ef73b29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:26:01.103025 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e083f-d181-4123-9e83-69ca0ef73b29 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:26:01.112794 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1455/2938] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:26:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 12:26:11.108642 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=77 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:26:15.001779 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25a099-6f44-4ed6-ad6f-fc0781fec89d None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:26:15.002378 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25a099-6f44-4ed6-ad6f-fc0781fec89d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:26:15.002378 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25a099-6f44-4ed6-ad6f-fc0781fec89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:26:15.022700 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-aa25a099-6f44-4ed6-ad6f-fc0781fec89d None None] MFA Rules not processed for user `33f480ef1b8b49df83276a66f445ae85`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:26:15.024037 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-aa25a099-6f44-4ed6-ad6f-fc0781fec89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:26:15.031920 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1484/2939] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 14 12:26:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 12:26:25.017602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:26:38.784402 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e083f-d181-4123-9e83-69ca0ef73b29 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:26:38.785840 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-118e083f-d181-4123-9e83-69ca0ef73b29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:26:38.842904 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e083f-d181-4123-9e83-69ca0ef73b29 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:26:38.846522 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278688ad-8af2-4a22-a10a-3c62ae9c484c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:26:38.847131 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278688ad-8af2-4a22-a10a-3c62ae9c484c service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:26:38.847416 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278688ad-8af2-4a22-a10a-3c62ae9c484c service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:26:38.847692 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278688ad-8af2-4a22-a10a-3c62ae9c484c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:26:38.849423 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-278688ad-8af2-4a22-a10a-3c62ae9c484c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:26:38.912206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278688ad-8af2-4a22-a10a-3c62ae9c484c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:26:38.913631 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278688ad-8af2-4a22-a10a-3c62ae9c484c service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:26:38.923569 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1456/2940] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:26:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:26:40.918820 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25a099-6f44-4ed6-ad6f-fc0781fec89d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:26:40.922351 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-aa25a099-6f44-4ed6-ad6f-fc0781fec89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:26:41.023739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25a099-6f44-4ed6-ad6f-fc0781fec89d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:26:41.029536 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1a76e-1de5-4bdc-92b8-1e4042b120eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:26:41.031744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1a76e-1de5-4bdc-92b8-1e4042b120eb service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:26:41.031744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1a76e-1de5-4bdc-92b8-1e4042b120eb service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:26:41.031744 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1a76e-1de5-4bdc-92b8-1e4042b120eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:26:41.033133 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-92a1a76e-1de5-4bdc-92b8-1e4042b120eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:26:41.046276 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1a76e-1de5-4bdc-92b8-1e4042b120eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:26:41.047963 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1a76e-1de5-4bdc-92b8-1e4042b120eb service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:26:41.058939 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1485/2941] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:26:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:26:48.918786 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=50 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:26:51.054320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=46 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:26:55.655502 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278688ad-8af2-4a22-a10a-3c62ae9c484c service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:26:55.657386 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-278688ad-8af2-4a22-a10a-3c62ae9c484c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:26:55.669009 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278688ad-8af2-4a22-a10a-3c62ae9c484c service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:26:55.672864 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0ab41-6dcf-43f0-9bc1-01cb6831e038 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:26:55.673474 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0ab41-6dcf-43f0-9bc1-01cb6831e038 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:26:55.673758 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0ab41-6dcf-43f0-9bc1-01cb6831e038 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:26:55.674076 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0ab41-6dcf-43f0-9bc1-01cb6831e038 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:26:55.675813 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b2a0ab41-6dcf-43f0-9bc1-01cb6831e038 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:26:55.690299 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0ab41-6dcf-43f0-9bc1-01cb6831e038 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:26:55.691742 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0ab41-6dcf-43f0-9bc1-01cb6831e038 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:26:55.703499 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1457/2942] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:26:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:27:02.758506 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1a76e-1de5-4bdc-92b8-1e4042b120eb service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:27:02.768477 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1a76e-1de5-4bdc-92b8-1e4042b120eb service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:27:02.772220 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995aa111-8e8b-43b3-bdb0-e8a116587331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:27:02.772940 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995aa111-8e8b-43b3-bdb0-e8a116587331 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:27:02.773228 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995aa111-8e8b-43b3-bdb0-e8a116587331 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:27:02.773506 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995aa111-8e8b-43b3-bdb0-e8a116587331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:27:02.776049 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-995aa111-8e8b-43b3-bdb0-e8a116587331 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:27:02.792284 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995aa111-8e8b-43b3-bdb0-e8a116587331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:27:02.793800 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995aa111-8e8b-43b3-bdb0-e8a116587331 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:27:02.805937 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1486/2943] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:27:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:27:05.697917 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=6 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:27:06.367467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc074ba-3220-4d24-a751-af49422691d2 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:27:06.368993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc074ba-3220-4d24-a751-af49422691d2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:27:06.368993 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc074ba-3220-4d24-a751-af49422691d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:27:06.430045 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-edc074ba-3220-4d24-a751-af49422691d2 None None] MFA Rules not processed for user `d76a164ed7e14d9eb2db016b68a84d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 12:27:06.431258 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-edc074ba-3220-4d24-a751-af49422691d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:27:06.495112 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1458/2944] 10.222.0.62 () {58 vars in 967 bytes} [Wed Sep 14 12:27:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 12:27:06.536466 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995aa111-8e8b-43b3-bdb0-e8a116587331 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:27:06.537900 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-995aa111-8e8b-43b3-bdb0-e8a116587331 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:27:06.630176 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995aa111-8e8b-43b3-bdb0-e8a116587331 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:27:06.635231 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d1cb5-0f32-4a5f-b6bc-ec1732d87cd1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1047c7d18c1466d989cd895117cd807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:27:06.635836 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d1cb5-0f32-4a5f-b6bc-ec1732d87cd1 service swift] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:27:06.636122 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d1cb5-0f32-4a5f-b6bc-ec1732d87cd1 service swift] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:27:06.636394 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d1cb5-0f32-4a5f-b6bc-ec1732d87cd1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:27:06.645343 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d1cb5-0f32-4a5f-b6bc-ec1732d87cd1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:27:06.646949 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d1cb5-0f32-4a5f-b6bc-ec1732d87cd1 service swift] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:27:06.654961 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1487/2945] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 14 12:27:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 12:27:12.523019 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc074ba-3220-4d24-a751-af49422691d2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:27:12.533299 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc074ba-3220-4d24-a751-af49422691d2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:27:12.537347 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0628627-204f-48bd-a708-a0503f197ced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:27:12.538012 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0628627-204f-48bd-a708-a0503f197ced service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:27:12.538290 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0628627-204f-48bd-a708-a0503f197ced service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:27:12.538792 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0628627-204f-48bd-a708-a0503f197ced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:27:12.541225 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c0628627-204f-48bd-a708-a0503f197ced service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:27:12.563323 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0628627-204f-48bd-a708-a0503f197ced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:27:12.565356 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0628627-204f-48bd-a708-a0503f197ced service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:27:12.574765 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1459/2946] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:27:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:27:15.368613 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d1cb5-0f32-4a5f-b6bc-ec1732d87cd1 service swift] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:27:15.370441 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-388d1cb5-0f32-4a5f-b6bc-ec1732d87cd1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:27:15.438513 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d1cb5-0f32-4a5f-b6bc-ec1732d87cd1 service swift] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:27:15.443251 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db4efd-7a6b-4b06-b658-dfb9d7422c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:27:15.443886 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db4efd-7a6b-4b06-b658-dfb9d7422c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:27:15.444172 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db4efd-7a6b-4b06-b658-dfb9d7422c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:27:15.444510 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db4efd-7a6b-4b06-b658-dfb9d7422c57 admin admin] PATH_INFO: `/v3/users/6e9703bf61d14dc4ac52a7c50fb8390e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:27:15.445930 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db4efd-7a6b-4b06-b658-dfb9d7422c57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e9703bf61d14dc4ac52a7c50fb8390e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:27:15.447887 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db4efd-7a6b-4b06-b658-dfb9d7422c57 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:27:15.636225 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a8db4efd-7a6b-4b06-b658-dfb9d7422c57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e9703bf61d14dc4ac52a7c50fb8390e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:15.645075 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:27:15.649414 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a8db4efd-7a6b-4b06-b658-dfb9d7422c57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e9703bf61d14dc4ac52a7c50fb8390e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:15.656317 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a8db4efd-7a6b-4b06-b658-dfb9d7422c57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e9703bf61d14dc4ac52a7c50fb8390e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:15.731183 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a8db4efd-7a6b-4b06-b658-dfb9d7422c57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e9703bf61d14dc4ac52a7c50fb8390e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:15.731841 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a8db4efd-7a6b-4b06-b658-dfb9d7422c57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e9703bf61d14dc4ac52a7c50fb8390e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:15.738521 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1488/2947] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:27:15 2022] DELETE /identity/v3/users/6e9703bf61d14dc4ac52a7c50fb8390e => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:27:16.009604 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0628627-204f-48bd-a708-a0503f197ced service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:27:16.021884 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0628627-204f-48bd-a708-a0503f197ced service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:27:16.079440 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128ec0f-4d11-45d9-be17-2233262a1456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:27:16.080625 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128ec0f-4d11-45d9-be17-2233262a1456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:27:16.080925 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128ec0f-4d11-45d9-be17-2233262a1456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:27:16.081200 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128ec0f-4d11-45d9-be17-2233262a1456 admin admin] PATH_INFO: `/v3/projects/b9a967a1f3444a1e8b7f4321722b5949` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:27:16.082364 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128ec0f-4d11-45d9-be17-2233262a1456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9a967a1f3444a1e8b7f4321722b5949)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:27:16.084253 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128ec0f-4d11-45d9-be17-2233262a1456 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:27:16.197956 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7128ec0f-4d11-45d9-be17-2233262a1456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9a967a1f3444a1e8b7f4321722b5949', 'request_id': 'req-7128ec0f-4d11-45d9-be17-2233262a1456'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:16.200448 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:27:16.322615 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7128ec0f-4d11-45d9-be17-2233262a1456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9a967a1f3444a1e8b7f4321722b5949', 'request_id': 'req-7128ec0f-4d11-45d9-be17-2233262a1456'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:16.328652 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7128ec0f-4d11-45d9-be17-2233262a1456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9a967a1f3444a1e8b7f4321722b5949', 'request_id': 'req-7128ec0f-4d11-45d9-be17-2233262a1456'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:16.340333 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7128ec0f-4d11-45d9-be17-2233262a1456 admin admin] The token cache is being invalidate because project b9a967a1f3444a1e8b7f4321722b5949 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:27:16.340626 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7128ec0f-4d11-45d9-be17-2233262a1456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:16.341272 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7128ec0f-4d11-45d9-be17-2233262a1456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:16.342778 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1460/2948] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:27:16 2022] DELETE /identity/v3/projects/b9a967a1f3444a1e8b7f4321722b5949 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:27:16.346916 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db4efd-7a6b-4b06-b658-dfb9d7422c57 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:27:16.348421 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a8db4efd-7a6b-4b06-b658-dfb9d7422c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:27:16.361604 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db4efd-7a6b-4b06-b658-dfb9d7422c57 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:27:16.424544 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d8e27-3a36-4bd4-aa1b-4cad26b0a2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:27:16.425249 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d8e27-3a36-4bd4-aa1b-4cad26b0a2ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:27:16.426473 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d8e27-3a36-4bd4-aa1b-4cad26b0a2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:27:16.426760 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d8e27-3a36-4bd4-aa1b-4cad26b0a2ba admin admin] PATH_INFO: `/v3/users/001a3299f26643c6b9c13e80a21f3eba` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:27:16.428323 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d8e27-3a36-4bd4-aa1b-4cad26b0a2ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=001a3299f26643c6b9c13e80a21f3eba)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:27:16.430444 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d8e27-3a36-4bd4-aa1b-4cad26b0a2ba admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:27:16.642504 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a20d8e27-3a36-4bd4-aa1b-4cad26b0a2ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '001a3299f26643c6b9c13e80a21f3eba'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:16.654459 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a20d8e27-3a36-4bd4-aa1b-4cad26b0a2ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '001a3299f26643c6b9c13e80a21f3eba'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:16.661320 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a20d8e27-3a36-4bd4-aa1b-4cad26b0a2ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '001a3299f26643c6b9c13e80a21f3eba'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:16.733520 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a20d8e27-3a36-4bd4-aa1b-4cad26b0a2ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '001a3299f26643c6b9c13e80a21f3eba'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:16.734133 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a20d8e27-3a36-4bd4-aa1b-4cad26b0a2ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '001a3299f26643c6b9c13e80a21f3eba'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:16.741062 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1489/2949] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:27:16 2022] DELETE /identity/v3/users/001a3299f26643c6b9c13e80a21f3eba => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:27:17.005385 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128ec0f-4d11-45d9-be17-2233262a1456 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:27:17.006895 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7128ec0f-4d11-45d9-be17-2233262a1456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:27:17.020894 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128ec0f-4d11-45d9-be17-2233262a1456 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:27:17.080297 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797790a0-b49a-42dc-a38f-413897800dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:27:17.080938 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797790a0-b49a-42dc-a38f-413897800dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:27:17.081221 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797790a0-b49a-42dc-a38f-413897800dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:27:17.081497 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797790a0-b49a-42dc-a38f-413897800dde admin admin] PATH_INFO: `/v3/projects/fa63592046c24d13a72f02371ee4f417` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:27:17.082655 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797790a0-b49a-42dc-a38f-413897800dde admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa63592046c24d13a72f02371ee4f417)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:27:17.084589 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797790a0-b49a-42dc-a38f-413897800dde admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:27:17.194616 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-797790a0-b49a-42dc-a38f-413897800dde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa63592046c24d13a72f02371ee4f417', 'request_id': 'req-797790a0-b49a-42dc-a38f-413897800dde'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:17.273681 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-797790a0-b49a-42dc-a38f-413897800dde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa63592046c24d13a72f02371ee4f417', 'request_id': 'req-797790a0-b49a-42dc-a38f-413897800dde'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:17.280037 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-797790a0-b49a-42dc-a38f-413897800dde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa63592046c24d13a72f02371ee4f417', 'request_id': 'req-797790a0-b49a-42dc-a38f-413897800dde'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:17.292227 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-797790a0-b49a-42dc-a38f-413897800dde admin admin] The token cache is being invalidate because project fa63592046c24d13a72f02371ee4f417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:27:17.292523 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-797790a0-b49a-42dc-a38f-413897800dde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:17.293175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-797790a0-b49a-42dc-a38f-413897800dde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:27:17.294696 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1461/2950] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:27:17 2022] DELETE /identity/v3/projects/fa63592046c24d13a72f02371ee4f417 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:27:26.736851 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=61,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:27:27.289212 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:DELETE=10,keystone:SELECT=57,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:28:47.600602 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d8e27-3a36-4bd4-aa1b-4cad26b0a2ba admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:28:47.602123 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a20d8e27-3a36-4bd4-aa1b-4cad26b0a2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:28:47.684739 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d8e27-3a36-4bd4-aa1b-4cad26b0a2ba admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:28:47.764707 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b226671-cd37-47a8-9ade-c1175019cb35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b093bc4c1d4ebcbd2a6bb16b70c029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:28:47.765623 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b226671-cd37-47a8-9ade-c1175019cb35 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:28:47.766576 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b226671-cd37-47a8-9ade-c1175019cb35 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:28:47.767030 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b226671-cd37-47a8-9ade-c1175019cb35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:28:47.769106 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9b226671-cd37-47a8-9ade-c1175019cb35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:28:47.793876 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b226671-cd37-47a8-9ade-c1175019cb35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:28:47.795549 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b226671-cd37-47a8-9ade-c1175019cb35 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:28:47.816642 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1490/2951] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:28:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:28:57.802154 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=62 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:28:59.413611 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797790a0-b49a-42dc-a38f-413897800dde admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:28:59.415096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-797790a0-b49a-42dc-a38f-413897800dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:28:59.472558 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797790a0-b49a-42dc-a38f-413897800dde admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:28:59.476115 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8dd619-79ad-4479-9e5a-12b893792191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13f1840d92d94c3ebd16000a4889c377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:28:59.476716 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8dd619-79ad-4479-9e5a-12b893792191 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:28:59.476999 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8dd619-79ad-4479-9e5a-12b893792191 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:28:59.477274 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8dd619-79ad-4479-9e5a-12b893792191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:28:59.479060 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-db8dd619-79ad-4479-9e5a-12b893792191 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:28:59.540837 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8dd619-79ad-4479-9e5a-12b893792191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:28:59.542306 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8dd619-79ad-4479-9e5a-12b893792191 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:28:59.552228 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1462/2952] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:28:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:29:06.987413 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b226671-cd37-47a8-9ade-c1175019cb35 service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:29:06.988936 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9b226671-cd37-47a8-9ade-c1175019cb35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:29:07.003390 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b226671-cd37-47a8-9ade-c1175019cb35 service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:29:07.008321 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf09c68-4b5c-4093-a8da-dbf3e0714d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:29:07.013100 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf09c68-4b5c-4093-a8da-dbf3e0714d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:29:07.013100 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf09c68-4b5c-4093-a8da-dbf3e0714d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:29:07.013100 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf09c68-4b5c-4093-a8da-dbf3e0714d26 admin admin] PATH_INFO: `/v3/users/33f480ef1b8b49df83276a66f445ae85` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:29:07.013100 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf09c68-4b5c-4093-a8da-dbf3e0714d26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33f480ef1b8b49df83276a66f445ae85)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:29:07.013100 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf09c68-4b5c-4093-a8da-dbf3e0714d26 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:29:07.189796 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2bf09c68-4b5c-4093-a8da-dbf3e0714d26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33f480ef1b8b49df83276a66f445ae85'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:07.202736 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2bf09c68-4b5c-4093-a8da-dbf3e0714d26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33f480ef1b8b49df83276a66f445ae85'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:07.210547 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2bf09c68-4b5c-4093-a8da-dbf3e0714d26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33f480ef1b8b49df83276a66f445ae85'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:07.289315 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2bf09c68-4b5c-4093-a8da-dbf3e0714d26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33f480ef1b8b49df83276a66f445ae85'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:07.289906 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2bf09c68-4b5c-4093-a8da-dbf3e0714d26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33f480ef1b8b49df83276a66f445ae85'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:07.296418 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1491/2953] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:29:06 2022] DELETE /identity/v3/users/33f480ef1b8b49df83276a66f445ae85 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:29:07.559206 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8dd619-79ad-4479-9e5a-12b893792191 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:29:07.569925 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8dd619-79ad-4479-9e5a-12b893792191 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:29:07.630507 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97e2de-9e9b-49a7-98cb-d34ff6e42795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:29:07.631142 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97e2de-9e9b-49a7-98cb-d34ff6e42795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:29:07.631425 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97e2de-9e9b-49a7-98cb-d34ff6e42795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:29:07.631708 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97e2de-9e9b-49a7-98cb-d34ff6e42795 admin admin] PATH_INFO: `/v3/projects/e0cdab1c080d4132adc6cd64d8627c68` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:29:07.632824 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97e2de-9e9b-49a7-98cb-d34ff6e42795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0cdab1c080d4132adc6cd64d8627c68)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:29:07.634769 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97e2de-9e9b-49a7-98cb-d34ff6e42795 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:29:07.779923 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2f97e2de-9e9b-49a7-98cb-d34ff6e42795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0cdab1c080d4132adc6cd64d8627c68', 'request_id': 'req-2f97e2de-9e9b-49a7-98cb-d34ff6e42795'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:07.783096 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:29:07.869186 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2f97e2de-9e9b-49a7-98cb-d34ff6e42795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0cdab1c080d4132adc6cd64d8627c68', 'request_id': 'req-2f97e2de-9e9b-49a7-98cb-d34ff6e42795'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:07.875363 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2f97e2de-9e9b-49a7-98cb-d34ff6e42795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0cdab1c080d4132adc6cd64d8627c68', 'request_id': 'req-2f97e2de-9e9b-49a7-98cb-d34ff6e42795'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:07.887018 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2f97e2de-9e9b-49a7-98cb-d34ff6e42795 admin admin] The token cache is being invalidate because project e0cdab1c080d4132adc6cd64d8627c68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:29:07.888340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2f97e2de-9e9b-49a7-98cb-d34ff6e42795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:07.888340 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2f97e2de-9e9b-49a7-98cb-d34ff6e42795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:07.889851 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1463/2954] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:29:07 2022] DELETE /identity/v3/projects/e0cdab1c080d4132adc6cd64d8627c68 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:29:07.894030 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf09c68-4b5c-4093-a8da-dbf3e0714d26 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:29:07.895472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2bf09c68-4b5c-4093-a8da-dbf3e0714d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:29:07.909970 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf09c68-4b5c-4093-a8da-dbf3e0714d26 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:29:07.970237 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a4fdb-d20b-44ed-a0ad-2ba9b518e0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:29:07.971068 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a4fdb-d20b-44ed-a0ad-2ba9b518e0b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:29:07.971472 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a4fdb-d20b-44ed-a0ad-2ba9b518e0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:29:07.971898 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a4fdb-d20b-44ed-a0ad-2ba9b518e0b6 admin admin] PATH_INFO: `/v3/users/2458d329b68d4a44814293fb42fe0eda` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:29:07.973152 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a4fdb-d20b-44ed-a0ad-2ba9b518e0b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2458d329b68d4a44814293fb42fe0eda)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:29:07.975144 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a4fdb-d20b-44ed-a0ad-2ba9b518e0b6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:29:08.190408 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1b6a4fdb-d20b-44ed-a0ad-2ba9b518e0b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2458d329b68d4a44814293fb42fe0eda'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:08.203252 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1b6a4fdb-d20b-44ed-a0ad-2ba9b518e0b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2458d329b68d4a44814293fb42fe0eda'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:08.210778 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1b6a4fdb-d20b-44ed-a0ad-2ba9b518e0b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2458d329b68d4a44814293fb42fe0eda'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:08.212445 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:29:08.306294 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1b6a4fdb-d20b-44ed-a0ad-2ba9b518e0b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2458d329b68d4a44814293fb42fe0eda'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:08.306950 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-1b6a4fdb-d20b-44ed-a0ad-2ba9b518e0b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2458d329b68d4a44814293fb42fe0eda'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:08.314155 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1492/2955] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 14 12:29:07 2022] DELETE /identity/v3/users/2458d329b68d4a44814293fb42fe0eda => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:29:08.573834 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97e2de-9e9b-49a7-98cb-d34ff6e42795 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:29:08.575367 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2f97e2de-9e9b-49a7-98cb-d34ff6e42795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:29:08.588179 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97e2de-9e9b-49a7-98cb-d34ff6e42795 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:29:08.647899 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee2dc0-26a2-493f-8812-2196acfbbc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2df9ec255e48ec9c7b4291f711f0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a95d34dd28804fc3b45b9488a19047b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:29:08.648547 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee2dc0-26a2-493f-8812-2196acfbbc6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:29:08.648839 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee2dc0-26a2-493f-8812-2196acfbbc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:29:08.649119 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee2dc0-26a2-493f-8812-2196acfbbc6e admin admin] PATH_INFO: `/v3/projects/c7041a5e4c714f5fa5b77e8b2c3af41b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:29:08.650298 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee2dc0-26a2-493f-8812-2196acfbbc6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7041a5e4c714f5fa5b77e8b2c3af41b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:29:08.652205 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee2dc0-26a2-493f-8812-2196acfbbc6e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:29:08.762832 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c8ee2dc0-26a2-493f-8812-2196acfbbc6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7041a5e4c714f5fa5b77e8b2c3af41b', 'request_id': 'req-c8ee2dc0-26a2-493f-8812-2196acfbbc6e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:08.837755 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c8ee2dc0-26a2-493f-8812-2196acfbbc6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7041a5e4c714f5fa5b77e8b2c3af41b', 'request_id': 'req-c8ee2dc0-26a2-493f-8812-2196acfbbc6e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:08.844066 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c8ee2dc0-26a2-493f-8812-2196acfbbc6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7041a5e4c714f5fa5b77e8b2c3af41b', 'request_id': 'req-c8ee2dc0-26a2-493f-8812-2196acfbbc6e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:08.856175 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c8ee2dc0-26a2-493f-8812-2196acfbbc6e admin admin] The token cache is being invalidate because project c7041a5e4c714f5fa5b77e8b2c3af41b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 12:29:08.856467 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c8ee2dc0-26a2-493f-8812-2196acfbbc6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:08.857124 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c8ee2dc0-26a2-493f-8812-2196acfbbc6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 12:29:08.858625 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1464/2956] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 14 12:29:08 2022] DELETE /identity/v3/projects/c7041a5e4c714f5fa5b77e8b2c3af41b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 12:29:18.374568 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:29:18.852995 dv-cinder-844962-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:DELETE=10,keystone:SELECT=57,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 12:30:17.733252 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a4fdb-d20b-44ed-a0ad-2ba9b518e0b6 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 12:30:17.735413 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1b6a4fdb-d20b-44ed-a0ad-2ba9b518e0b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:30:17.812110 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a4fdb-d20b-44ed-a0ad-2ba9b518e0b6 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 12:30:17.883072 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ec320-3987-473b-9bd0-31e2ea30b0d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca10dd7ed714edb8b87017deecc41b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '449a5fac1acd478abea544446c08ecd9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 12:30:17.883706 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ec320-3987-473b-9bd0-31e2ea30b0d2 service placement] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 12:30:17.883999 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ec320-3987-473b-9bd0-31e2ea30b0d2 service placement] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 12:30:17.884281 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ec320-3987-473b-9bd0-31e2ea30b0d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 12:30:17.886643 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b77ec320-3987-473b-9bd0-31e2ea30b0d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 12:30:17.900969 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ec320-3987-473b-9bd0-31e2ea30b0d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 12:30:17.902723 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ec320-3987-473b-9bd0-31e2ea30b0d2 service placement] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 12:30:17.915148 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1493/2957] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 14 12:30:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 12:30:27.909098 dv-cinder-844962-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=58 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}